Login
[x]
Log in using an account from:
Fedora Account System
Red Hat Associate
Red Hat Customer
Or login using a Red Hat Bugzilla account
Forgot Password
Login:
Hide Forgot
Create an Account
Red Hat Bugzilla – Attachment 895081 Details for
Bug 1084789
template of thin provision NFS,can't be copied to block data domains
[?]
New
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
|
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh83 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
This site requires JavaScript to be enabled to function correctly, please enable it.
another logs attachment
vdsm.log (text/x-log), 8.55 MB, created by
Ori Gofen
on 2014-05-13 10:04:41 UTC
(
hide
)
Description:
another logs attachment
Filename:
MIME Type:
Creator:
Ori Gofen
Created:
2014-05-13 10:04:41 UTC
Size:
8.55 MB
patch
obsolete
>BindingXMLRPC::ERROR::2014-05-13 11:01:02,301::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:01:02,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:02,639::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0390441 s, 26.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:01:04,652::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:05,013::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.316375 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:01:05,325::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:01:07,026::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:07,100::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0295683 s, 34.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:01:08,354::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:01:09,113::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:09,194::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363096 s, 28.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:01:10,858::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:01:10,866::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000604974 s, 6.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:01:11,074::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:01:11,074::task::595::TaskManager.Task::(_updateState) Task=`0d45fcc6-75fd-483c-a719-03d593f103b7`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:01:11,074::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:01:11,075::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:01:11,075::task::1185::TaskManager.Task::(prepare) Task=`0d45fcc6-75fd-483c-a719-03d593f103b7`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:01:11,075::task::595::TaskManager.Task::(_updateState) Task=`0d45fcc6-75fd-483c-a719-03d593f103b7`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:01:11,075::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:01:11,075::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:01:11,076::task::990::TaskManager.Task::(_decref) Task=`0d45fcc6-75fd-483c-a719-03d593f103b7`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:01:11,080::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:01:11,080::task::595::TaskManager.Task::(_updateState) Task=`4f2c223c-041b-4b4b-bc0e-efb0e61260b9`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:01:11,081::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:01:11,081::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0c3de814-9779-49a3-a491-1e6dfe9f4c78`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:01:11,081::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:01:11,082::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:01:11,082::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0c3de814-9779-49a3-a491-1e6dfe9f4c78`::Granted request >Thread-338::DEBUG::2014-05-13 11:01:11,082::task::827::TaskManager.Task::(resourceAcquired) Task=`4f2c223c-041b-4b4b-bc0e-efb0e61260b9`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:01:11,083::task::990::TaskManager.Task::(_decref) Task=`4f2c223c-041b-4b4b-bc0e-efb0e61260b9`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:01:11,084::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:01:11,084::task::1185::TaskManager.Task::(prepare) Task=`4f2c223c-041b-4b4b-bc0e-efb0e61260b9`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:01:11,084::task::595::TaskManager.Task::(_updateState) Task=`4f2c223c-041b-4b4b-bc0e-efb0e61260b9`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:01:11,084::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:01:11,085::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:01:11,085::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:01:11,085::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:01:11,085::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:01:11,086::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:01:11,086::task::990::TaskManager.Task::(_decref) Task=`4f2c223c-041b-4b4b-bc0e-efb0e61260b9`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:01:11,207::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:11,285::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0350042 s, 29.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:01:11,373::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:01:12,511::task::595::TaskManager.Task::(_updateState) Task=`fb2cf3c1-5484-43b6-aea4-d6404dfad39d`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:01:12,511::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:01:12,511::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000604974', 'lastCheck': '1.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:01:12,511::task::1185::TaskManager.Task::(prepare) Task=`fb2cf3c1-5484-43b6-aea4-d6404dfad39d`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000604974', 'lastCheck': '1.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:01:12,512::task::595::TaskManager.Task::(_updateState) Task=`fb2cf3c1-5484-43b6-aea4-d6404dfad39d`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:01:12,512::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:01:12,512::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:01:12,512::task::990::TaskManager.Task::(_decref) Task=`fb2cf3c1-5484-43b6-aea4-d6404dfad39d`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:01:13,298::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:13,374::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0308844 s, 33.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:01:14,392::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:01:15,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:15,749::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.317374 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:01:17,411::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:01:17,762::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:17,842::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352524 s, 29.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:01:19,855::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:19,929::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313012 s, 32.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:01:20,430::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:01:20,874::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:01:20,882::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000674252 s, 6.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:01:21,106::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:01:21,106::task::595::TaskManager.Task::(_updateState) Task=`796b6af8-c1aa-42b0-bb3c-85510e02fa73`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:01:21,106::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:01:21,107::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:01:21,107::task::1185::TaskManager.Task::(prepare) Task=`796b6af8-c1aa-42b0-bb3c-85510e02fa73`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:01:21,107::task::595::TaskManager.Task::(_updateState) Task=`796b6af8-c1aa-42b0-bb3c-85510e02fa73`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:01:21,107::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:01:21,107::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:01:21,108::task::990::TaskManager.Task::(_decref) Task=`796b6af8-c1aa-42b0-bb3c-85510e02fa73`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:01:21,112::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:01:21,112::task::595::TaskManager.Task::(_updateState) Task=`8e13b2af-045a-4d5c-819c-cdda438293d6`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:01:21,113::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:01:21,113::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0853c5fc-4796-4f2b-a947-4c0d2f2700e8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:01:21,113::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:01:21,114::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:01:21,114::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0853c5fc-4796-4f2b-a947-4c0d2f2700e8`::Granted request >Thread-338::DEBUG::2014-05-13 11:01:21,114::task::827::TaskManager.Task::(resourceAcquired) Task=`8e13b2af-045a-4d5c-819c-cdda438293d6`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:01:21,114::task::990::TaskManager.Task::(_decref) Task=`8e13b2af-045a-4d5c-819c-cdda438293d6`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:01:21,115::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:01:21,115::task::1185::TaskManager.Task::(prepare) Task=`8e13b2af-045a-4d5c-819c-cdda438293d6`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:01:21,115::task::595::TaskManager.Task::(_updateState) Task=`8e13b2af-045a-4d5c-819c-cdda438293d6`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:01:21,116::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:01:21,116::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:01:21,116::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:01:21,116::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:01:21,116::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:01:21,116::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:01:21,117::task::990::TaskManager.Task::(_decref) Task=`8e13b2af-045a-4d5c-819c-cdda438293d6`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:01:21,942::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:22,018::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031158 s, 32.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:01:23,453::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:01:24,031::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:24,109::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0350985 s, 29.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:01:26,122::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:26,201::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0354031 s, 28.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:01:26,487::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:01:27,642::task::595::TaskManager.Task::(_updateState) Task=`3695e68c-db94-411f-b522-a516fcc8e052`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:01:27,642::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:01:27,643::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000674252', 'lastCheck': '6.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:01:27,643::task::1185::TaskManager.Task::(prepare) Task=`3695e68c-db94-411f-b522-a516fcc8e052`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000674252', 'lastCheck': '6.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:01:27,643::task::595::TaskManager.Task::(_updateState) Task=`3695e68c-db94-411f-b522-a516fcc8e052`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:01:27,643::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:01:27,643::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:01:27,643::task::990::TaskManager.Task::(_decref) Task=`3695e68c-db94-411f-b522-a516fcc8e052`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:01:28,214::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:28,576::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.319638 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:01:29,504::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:01:30,589::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:30,663::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315598 s, 32.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:01:30,889::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:01:30,898::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000656293 s, 6.2 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:01:31,134::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:01:31,134::task::595::TaskManager.Task::(_updateState) Task=`9b88c751-6d16-4257-a4ce-aab8d2341c08`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:01:31,134::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:01:31,135::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:01:31,135::task::1185::TaskManager.Task::(prepare) Task=`9b88c751-6d16-4257-a4ce-aab8d2341c08`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:01:31,135::task::595::TaskManager.Task::(_updateState) Task=`9b88c751-6d16-4257-a4ce-aab8d2341c08`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:01:31,135::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:01:31,135::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:01:31,136::task::990::TaskManager.Task::(_decref) Task=`9b88c751-6d16-4257-a4ce-aab8d2341c08`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:01:31,142::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:01:31,143::task::595::TaskManager.Task::(_updateState) Task=`a1a1dbcf-2f44-48e0-b946-64a6b08ab8bd`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:01:31,143::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:01:31,143::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c3a3ab42-295b-4d75-95fa-49a7ad26d039`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:01:31,144::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:01:31,144::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:01:31,144::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c3a3ab42-295b-4d75-95fa-49a7ad26d039`::Granted request >Thread-338::DEBUG::2014-05-13 11:01:31,144::task::827::TaskManager.Task::(resourceAcquired) Task=`a1a1dbcf-2f44-48e0-b946-64a6b08ab8bd`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:01:31,145::task::990::TaskManager.Task::(_decref) Task=`a1a1dbcf-2f44-48e0-b946-64a6b08ab8bd`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:01:31,145::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:01:31,145::task::1185::TaskManager.Task::(prepare) Task=`a1a1dbcf-2f44-48e0-b946-64a6b08ab8bd`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:01:31,146::task::595::TaskManager.Task::(_updateState) Task=`a1a1dbcf-2f44-48e0-b946-64a6b08ab8bd`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:01:31,146::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:01:31,146::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:01:31,146::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:01:31,147::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:01:31,147::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:01:31,147::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:01:31,147::task::990::TaskManager.Task::(_decref) Task=`a1a1dbcf-2f44-48e0-b946-64a6b08ab8bd`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:01:32,520::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:01:32,676::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:32,757::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359526 s, 28.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:01:34,769::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:34,847::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0329025 s, 31.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:01:35,539::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:01:36,860::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:36,936::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319986 s, 32.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:01:38,557::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:01:38,949::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:39,034::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0436007 s, 23.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:01:40,905::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:01:40,914::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00126704 s, 3.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:01:41,047::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:41,139::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0472925 s, 21.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:01:41,168::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:01:41,169::task::595::TaskManager.Task::(_updateState) Task=`334fba67-f043-42ca-bb49-503c87e35a16`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:01:41,169::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:01:41,169::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:01:41,169::task::1185::TaskManager.Task::(prepare) Task=`334fba67-f043-42ca-bb49-503c87e35a16`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:01:41,170::task::595::TaskManager.Task::(_updateState) Task=`334fba67-f043-42ca-bb49-503c87e35a16`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:01:41,170::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:01:41,170::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:01:41,170::task::990::TaskManager.Task::(_decref) Task=`334fba67-f043-42ca-bb49-503c87e35a16`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:01:41,175::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:01:41,175::task::595::TaskManager.Task::(_updateState) Task=`fc3ab014-db46-4110-824d-74748b0bdc4e`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:01:41,175::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:01:41,176::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`68380df0-881c-4b34-aefc-0904967a5560`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:01:41,176::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:01:41,176::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:01:41,177::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`68380df0-881c-4b34-aefc-0904967a5560`::Granted request >Thread-338::DEBUG::2014-05-13 11:01:41,177::task::827::TaskManager.Task::(resourceAcquired) Task=`fc3ab014-db46-4110-824d-74748b0bdc4e`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:01:41,177::task::990::TaskManager.Task::(_decref) Task=`fc3ab014-db46-4110-824d-74748b0bdc4e`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:01:41,178::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:01:41,178::task::1185::TaskManager.Task::(prepare) Task=`fc3ab014-db46-4110-824d-74748b0bdc4e`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:01:41,178::task::595::TaskManager.Task::(_updateState) Task=`fc3ab014-db46-4110-824d-74748b0bdc4e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:01:41,178::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:01:41,179::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:01:41,179::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:01:41,179::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:01:41,179::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:01:41,179::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:01:41,180::task::990::TaskManager.Task::(_decref) Task=`fc3ab014-db46-4110-824d-74748b0bdc4e`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:01:41,576::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:01:42,771::task::595::TaskManager.Task::(_updateState) Task=`15c51b77-bf19-408f-b712-226e01ebd768`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:01:42,772::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:01:42,772::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00126704', 'lastCheck': '1.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:01:42,772::task::1185::TaskManager.Task::(prepare) Task=`15c51b77-bf19-408f-b712-226e01ebd768`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00126704', 'lastCheck': '1.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:01:42,772::task::595::TaskManager.Task::(_updateState) Task=`15c51b77-bf19-408f-b712-226e01ebd768`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:01:42,772::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:01:42,772::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:01:42,773::task::990::TaskManager.Task::(_decref) Task=`15c51b77-bf19-408f-b712-226e01ebd768`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:01:43,152::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:43,227::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309925 s, 33.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:01:44,595::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:01:45,240::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:45,323::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0397317 s, 25.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:01:47,336::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:47,411::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309828 s, 33.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:01:47,613::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:01:49,424::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:49,498::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303869 s, 33.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:01:50,633::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:01:50,921::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:01:50,930::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000631256 s, 6.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:01:51,197::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:01:51,198::task::595::TaskManager.Task::(_updateState) Task=`6a268ba5-9f98-48ca-b529-01eed0eba831`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:01:51,198::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:01:51,198::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:01:51,199::task::1185::TaskManager.Task::(prepare) Task=`6a268ba5-9f98-48ca-b529-01eed0eba831`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:01:51,199::task::595::TaskManager.Task::(_updateState) Task=`6a268ba5-9f98-48ca-b529-01eed0eba831`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:01:51,199::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:01:51,199::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:01:51,199::task::990::TaskManager.Task::(_decref) Task=`6a268ba5-9f98-48ca-b529-01eed0eba831`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:01:51,204::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:01:51,204::task::595::TaskManager.Task::(_updateState) Task=`c4fc5bf5-b395-4c20-9f6c-2f51edba2069`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:01:51,205::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:01:51,205::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`dfd34966-3b30-4d6b-80f9-6008404f1869`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:01:51,205::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:01:51,205::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:01:51,206::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`dfd34966-3b30-4d6b-80f9-6008404f1869`::Granted request >Thread-338::DEBUG::2014-05-13 11:01:51,206::task::827::TaskManager.Task::(resourceAcquired) Task=`c4fc5bf5-b395-4c20-9f6c-2f51edba2069`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:01:51,206::task::990::TaskManager.Task::(_decref) Task=`c4fc5bf5-b395-4c20-9f6c-2f51edba2069`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:01:51,207::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:01:51,207::task::1185::TaskManager.Task::(prepare) Task=`c4fc5bf5-b395-4c20-9f6c-2f51edba2069`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:01:51,207::task::595::TaskManager.Task::(_updateState) Task=`c4fc5bf5-b395-4c20-9f6c-2f51edba2069`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:01:51,207::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:01:51,208::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:01:51,208::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:01:51,208::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:01:51,208::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:01:51,208::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:01:51,209::task::990::TaskManager.Task::(_decref) Task=`c4fc5bf5-b395-4c20-9f6c-2f51edba2069`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:01:51,510::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:51,587::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0344374 s, 29.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:01:53,595::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:01:53,652::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:01:53,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305588 s, 33.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:01:55,686::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:55,776::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0464178 s, 22.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:01:56,671::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:01:57,789::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:57,864::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313837 s, 32.6 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:01:57,907::task::595::TaskManager.Task::(_updateState) Task=`d01d0f12-6217-4b1f-8bb0-d430b3ac1027`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:01:57,907::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:01:57,907::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000631256', 'lastCheck': '7.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:01:57,908::task::1185::TaskManager.Task::(prepare) Task=`d01d0f12-6217-4b1f-8bb0-d430b3ac1027`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000631256', 'lastCheck': '7.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:01:57,908::task::595::TaskManager.Task::(_updateState) Task=`d01d0f12-6217-4b1f-8bb0-d430b3ac1027`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:01:57,908::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:01:57,908::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:01:57,908::task::990::TaskManager.Task::(_decref) Task=`d01d0f12-6217-4b1f-8bb0-d430b3ac1027`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:01:59,689::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:01:59,877::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:01:59,959::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374428 s, 27.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:02:00,937::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:02:00,945::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000656595 s, 6.2 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:02:01,228::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:02:01,229::task::595::TaskManager.Task::(_updateState) Task=`42a52d89-ea55-4b58-b991-61419d285e1c`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:02:01,229::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:02:01,229::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:02:01,230::task::1185::TaskManager.Task::(prepare) Task=`42a52d89-ea55-4b58-b991-61419d285e1c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:02:01,230::task::595::TaskManager.Task::(_updateState) Task=`42a52d89-ea55-4b58-b991-61419d285e1c`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:02:01,230::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:02:01,230::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:02:01,230::task::990::TaskManager.Task::(_decref) Task=`42a52d89-ea55-4b58-b991-61419d285e1c`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:02:01,235::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:02:01,235::task::595::TaskManager.Task::(_updateState) Task=`cf3e3a19-8cc2-41e5-9d3e-7aae08a2123f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:02:01,235::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:02:01,236::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`88cfb8a6-41cb-46fb-bbab-bafe3abff36f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:02:01,236::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:02:01,236::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:02:01,236::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`88cfb8a6-41cb-46fb-bbab-bafe3abff36f`::Granted request >Thread-338::DEBUG::2014-05-13 11:02:01,237::task::827::TaskManager.Task::(resourceAcquired) Task=`cf3e3a19-8cc2-41e5-9d3e-7aae08a2123f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:02:01,237::task::990::TaskManager.Task::(_decref) Task=`cf3e3a19-8cc2-41e5-9d3e-7aae08a2123f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:02:01,238::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:02:01,238::task::1185::TaskManager.Task::(prepare) Task=`cf3e3a19-8cc2-41e5-9d3e-7aae08a2123f`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:02:01,238::task::595::TaskManager.Task::(_updateState) Task=`cf3e3a19-8cc2-41e5-9d3e-7aae08a2123f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:02:01,238::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:02:01,238::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:02:01,239::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:02:01,239::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:02:01,239::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:02:01,239::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:02:01,239::task::990::TaskManager.Task::(_decref) Task=`cf3e3a19-8cc2-41e5-9d3e-7aae08a2123f`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:02:01,972::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:02:02,050::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352762 s, 29.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:02:02,709::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:02:04,063::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:02:04,139::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311522 s, 32.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:02:05,726::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:02:06,152::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:02:06,233::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0371262 s, 27.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:02:08,246::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:02:08,320::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0302648 s, 33.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:02:08,745::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:02:10,333::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:02:10,703::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.324615 s, 3.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:02:10,952::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:02:10,961::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00112963 s, 3.6 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:02:11,260::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [45b9a91f] >Thread-66::DEBUG::2014-05-13 11:02:11,260::task::595::TaskManager.Task::(_updateState) Task=`f1472c74-e8aa-4719-b65e-7a34f408b743`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:02:11,260::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:02:11,261::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:02:11,261::task::1185::TaskManager.Task::(prepare) Task=`f1472c74-e8aa-4719-b65e-7a34f408b743`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:02:11,261::task::595::TaskManager.Task::(_updateState) Task=`f1472c74-e8aa-4719-b65e-7a34f408b743`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:02:11,261::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:02:11,261::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:02:11,261::task::990::TaskManager.Task::(_decref) Task=`f1472c74-e8aa-4719-b65e-7a34f408b743`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:02:11,266::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [45b9a91f] >Thread-338::DEBUG::2014-05-13 11:02:11,266::task::595::TaskManager.Task::(_updateState) Task=`cfe79459-a21d-4759-aeb4-09afbc31e24c`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:02:11,267::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:02:11,268::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4bfbe4ea-03c3-4c55-8435-03b27d559e7a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:02:11,268::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:02:11,269::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:02:11,269::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4bfbe4ea-03c3-4c55-8435-03b27d559e7a`::Granted request >Thread-338::DEBUG::2014-05-13 11:02:11,269::task::827::TaskManager.Task::(resourceAcquired) Task=`cfe79459-a21d-4759-aeb4-09afbc31e24c`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:02:11,269::task::990::TaskManager.Task::(_decref) Task=`cfe79459-a21d-4759-aeb4-09afbc31e24c`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:02:11,270::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:02:11,270::task::1185::TaskManager.Task::(prepare) Task=`cfe79459-a21d-4759-aeb4-09afbc31e24c`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:02:11,270::task::595::TaskManager.Task::(_updateState) Task=`cfe79459-a21d-4759-aeb4-09afbc31e24c`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:02:11,270::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:02:11,271::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:02:11,271::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:02:11,271::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:02:11,271::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:02:11,271::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:02:11,272::task::990::TaskManager.Task::(_decref) Task=`cfe79459-a21d-4759-aeb4-09afbc31e24c`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:02:11,765::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:02:12,716::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:02:12,790::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0300229 s, 34.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:02:13,041::task::595::TaskManager.Task::(_updateState) Task=`f640ec8e-03fd-4052-803a-997ca001cb78`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:02:13,042::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:02:13,042::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00112963', 'lastCheck': '2.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:02:13,042::task::1185::TaskManager.Task::(prepare) Task=`f640ec8e-03fd-4052-803a-997ca001cb78`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00112963', 'lastCheck': '2.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:02:13,042::task::595::TaskManager.Task::(_updateState) Task=`f640ec8e-03fd-4052-803a-997ca001cb78`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:02:13,042::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:02:13,043::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:02:13,043::task::990::TaskManager.Task::(_decref) Task=`f640ec8e-03fd-4052-803a-997ca001cb78`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:02:14,787::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:02:14,803::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:02:14,882::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362427 s, 28.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:02:16,896::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:02:16,977::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.038003 s, 26.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:02:17,811::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:02:18,990::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:02:19,066::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031103 s, 32.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:02:20,830::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:02:20,991::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:02:20,999::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000673762 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:02:21,079::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 11:02:21,290::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4478bd69] >Thread-66::DEBUG::2014-05-13 11:02:21,290::task::595::TaskManager.Task::(_updateState) Task=`ea401434-a652-457d-910b-4cb72c583efb`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:02:21,291::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:02:21,291::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:02:21,291::task::1185::TaskManager.Task::(prepare) Task=`ea401434-a652-457d-910b-4cb72c583efb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:02:21,291::task::595::TaskManager.Task::(_updateState) Task=`ea401434-a652-457d-910b-4cb72c583efb`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:02:21,291::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:02:21,292::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:02:21,292::task::990::TaskManager.Task::(_decref) Task=`ea401434-a652-457d-910b-4cb72c583efb`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:02:21,297::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4478bd69] >Thread-338::DEBUG::2014-05-13 11:02:21,297::task::595::TaskManager.Task::(_updateState) Task=`3a8eb98b-6513-4748-8e04-6b2089ad4707`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:02:21,297::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:02:21,298::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`953657df-7dcc-48a4-8673-ddd3b96935a0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:02:21,298::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:02:21,298::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:02:21,299::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`953657df-7dcc-48a4-8673-ddd3b96935a0`::Granted request >Thread-338::DEBUG::2014-05-13 11:02:21,299::task::827::TaskManager.Task::(resourceAcquired) Task=`3a8eb98b-6513-4748-8e04-6b2089ad4707`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:02:21,299::task::990::TaskManager.Task::(_decref) Task=`3a8eb98b-6513-4748-8e04-6b2089ad4707`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:02:21,300::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:02:21,300::task::1185::TaskManager.Task::(prepare) Task=`3a8eb98b-6513-4748-8e04-6b2089ad4707`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:02:21,300::task::595::TaskManager.Task::(_updateState) Task=`3a8eb98b-6513-4748-8e04-6b2089ad4707`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:02:21,300::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:02:21,301::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:02:21,301::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:02:21,301::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:02:21,301::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:02:21,301::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:02:21,302::task::990::TaskManager.Task::(_decref) Task=`3a8eb98b-6513-4748-8e04-6b2089ad4707`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:02:21,475::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.353188 s, 2.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:02:23,489::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:02:23,564::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312791 s, 32.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:02:23,850::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:02:25,577::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:02:25,651::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310868 s, 32.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:02:26,868::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:02:27,664::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:02:27,740::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321873 s, 31.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:02:28,178::task::595::TaskManager.Task::(_updateState) Task=`7c35e9e3-69e5-4291-b67d-0a5e1d48c0fc`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:02:28,179::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:02:28,179::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000673762', 'lastCheck': '7.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:02:28,179::task::1185::TaskManager.Task::(prepare) Task=`7c35e9e3-69e5-4291-b67d-0a5e1d48c0fc`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000673762', 'lastCheck': '7.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:02:28,179::task::595::TaskManager.Task::(_updateState) Task=`7c35e9e3-69e5-4291-b67d-0a5e1d48c0fc`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:02:28,179::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:02:28,180::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:02:28,180::task::990::TaskManager.Task::(_decref) Task=`7c35e9e3-69e5-4291-b67d-0a5e1d48c0fc`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:02:29,753::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:02:29,831::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0346845 s, 29.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:02:29,899::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:02:31,007::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:02:31,015::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00066252 s, 6.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:02:31,321::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [12ac301a] >Thread-13::DEBUG::2014-05-13 11:02:31,322::task::595::TaskManager.Task::(_updateState) Task=`5b62db5e-0a48-4094-b72c-61c5955b2435`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:02:31,322::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:02:31,322::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:02:31,322::task::1185::TaskManager.Task::(prepare) Task=`5b62db5e-0a48-4094-b72c-61c5955b2435`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:02:31,322::task::595::TaskManager.Task::(_updateState) Task=`5b62db5e-0a48-4094-b72c-61c5955b2435`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:02:31,323::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:02:31,323::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:02:31,323::task::990::TaskManager.Task::(_decref) Task=`5b62db5e-0a48-4094-b72c-61c5955b2435`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:02:31,538::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [12ac301a] >Thread-338::DEBUG::2014-05-13 11:02:31,538::task::595::TaskManager.Task::(_updateState) Task=`1e6bf01e-55ff-4caa-9bd5-8d6517aa97bf`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:02:31,539::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:02:31,539::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d35c7b6d-a9b2-4468-b49a-44c487892472`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:02:31,539::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:02:31,540::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:02:31,540::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d35c7b6d-a9b2-4468-b49a-44c487892472`::Granted request >Thread-338::DEBUG::2014-05-13 11:02:31,540::task::827::TaskManager.Task::(resourceAcquired) Task=`1e6bf01e-55ff-4caa-9bd5-8d6517aa97bf`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:02:31,540::task::990::TaskManager.Task::(_decref) Task=`1e6bf01e-55ff-4caa-9bd5-8d6517aa97bf`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:02:31,541::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:02:31,541::task::1185::TaskManager.Task::(prepare) Task=`1e6bf01e-55ff-4caa-9bd5-8d6517aa97bf`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:02:31,541::task::595::TaskManager.Task::(_updateState) Task=`1e6bf01e-55ff-4caa-9bd5-8d6517aa97bf`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:02:31,542::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:02:31,542::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:02:31,542::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:02:31,542::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:02:31,542::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:02:31,542::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:02:31,543::task::990::TaskManager.Task::(_decref) Task=`1e6bf01e-55ff-4caa-9bd5-8d6517aa97bf`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:02:31,844::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:02:31,927::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0376696 s, 27.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:02:32,921::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:02:33,940::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:02:34,351::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.367965 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:02:35,940::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:02:36,364::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:02:36,443::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359141 s, 28.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:02:38,456::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:02:38,546::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314014 s, 32.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:02:38,969::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:02:40,560::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:02:40,637::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316345 s, 32.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:02:41,022::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:02:41,031::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000598776 s, 6.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:02:41,563::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [28fd42b9] >Thread-66::DEBUG::2014-05-13 11:02:41,563::task::595::TaskManager.Task::(_updateState) Task=`ac9fcb63-670f-4601-92e3-315bd78825c6`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:02:41,564::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:02:41,564::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:02:41,564::task::1185::TaskManager.Task::(prepare) Task=`ac9fcb63-670f-4601-92e3-315bd78825c6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:02:41,564::task::595::TaskManager.Task::(_updateState) Task=`ac9fcb63-670f-4601-92e3-315bd78825c6`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:02:41,565::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:02:41,565::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:02:41,565::task::990::TaskManager.Task::(_decref) Task=`ac9fcb63-670f-4601-92e3-315bd78825c6`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:02:41,569::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [28fd42b9] >Thread-338::DEBUG::2014-05-13 11:02:41,570::task::595::TaskManager.Task::(_updateState) Task=`e04faf95-53f6-4010-8f89-8cee69a4f8f2`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:02:41,570::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:02:41,570::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`41d59f2c-3252-4970-b35c-35d29e81223b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:02:41,571::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:02:41,571::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:02:41,571::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`41d59f2c-3252-4970-b35c-35d29e81223b`::Granted request >Thread-338::DEBUG::2014-05-13 11:02:41,571::task::827::TaskManager.Task::(resourceAcquired) Task=`e04faf95-53f6-4010-8f89-8cee69a4f8f2`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:02:41,572::task::990::TaskManager.Task::(_decref) Task=`e04faf95-53f6-4010-8f89-8cee69a4f8f2`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:02:41,572::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:02:41,573::task::1185::TaskManager.Task::(prepare) Task=`e04faf95-53f6-4010-8f89-8cee69a4f8f2`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:02:41,573::task::595::TaskManager.Task::(_updateState) Task=`e04faf95-53f6-4010-8f89-8cee69a4f8f2`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:02:41,573::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:02:41,573::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:02:41,573::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:02:41,574::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:02:41,574::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:02:41,574::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:02:41,574::task::990::TaskManager.Task::(_decref) Task=`e04faf95-53f6-4010-8f89-8cee69a4f8f2`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:02:41,988::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:02:42,650::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:02:42,726::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311286 s, 32.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:02:43,595::task::595::TaskManager.Task::(_updateState) Task=`42ee2867-1c10-468e-99e5-7e85434a2749`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:02:43,595::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:02:43,595::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000598776', 'lastCheck': '2.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:02:43,595::task::1185::TaskManager.Task::(prepare) Task=`42ee2867-1c10-468e-99e5-7e85434a2749`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000598776', 'lastCheck': '2.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:02:43,596::task::595::TaskManager.Task::(_updateState) Task=`42ee2867-1c10-468e-99e5-7e85434a2749`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:02:43,596::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:02:43,596::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:02:43,596::task::990::TaskManager.Task::(_decref) Task=`42ee2867-1c10-468e-99e5-7e85434a2749`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:02:44,740::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:02:45,015::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:02:45,155::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.370991 s, 2.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:02:47,168::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:02:47,248::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0376688 s, 27.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:02:48,038::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:02:49,261::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:02:49,335::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306681 s, 33.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:02:51,058::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:02:51,076::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:02:51,085::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000669046 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:02:51,348::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:02:51,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0349684 s, 29.3 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:02:51,593::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3e537ec9] >Thread-66::DEBUG::2014-05-13 11:02:51,594::task::595::TaskManager.Task::(_updateState) Task=`f3de984e-cf53-4c45-a13a-ef5718d8d1bf`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:02:51,594::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:02:51,594::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:02:51,595::task::1185::TaskManager.Task::(prepare) Task=`f3de984e-cf53-4c45-a13a-ef5718d8d1bf`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:02:51,595::task::595::TaskManager.Task::(_updateState) Task=`f3de984e-cf53-4c45-a13a-ef5718d8d1bf`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:02:51,595::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:02:51,595::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:02:51,595::task::990::TaskManager.Task::(_decref) Task=`f3de984e-cf53-4c45-a13a-ef5718d8d1bf`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:02:51,600::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3e537ec9] >Thread-338::DEBUG::2014-05-13 11:02:51,600::task::595::TaskManager.Task::(_updateState) Task=`ed476e73-f09b-45bd-a492-d405c39315ff`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:02:51,600::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:02:51,601::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9d265cbf-d640-4d43-89a3-1dedfc041bc1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:02:51,601::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:02:51,601::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:02:51,601::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9d265cbf-d640-4d43-89a3-1dedfc041bc1`::Granted request >Thread-338::DEBUG::2014-05-13 11:02:51,602::task::827::TaskManager.Task::(resourceAcquired) Task=`ed476e73-f09b-45bd-a492-d405c39315ff`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:02:51,602::task::990::TaskManager.Task::(_decref) Task=`ed476e73-f09b-45bd-a492-d405c39315ff`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:02:51,603::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:02:51,603::task::1185::TaskManager.Task::(prepare) Task=`ed476e73-f09b-45bd-a492-d405c39315ff`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:02:51,603::task::595::TaskManager.Task::(_updateState) Task=`ed476e73-f09b-45bd-a492-d405c39315ff`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:02:51,603::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:02:51,603::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:02:51,604::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:02:51,604::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:02:51,604::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:02:51,604::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:02:51,604::task::990::TaskManager.Task::(_decref) Task=`ed476e73-f09b-45bd-a492-d405c39315ff`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:02:53,440::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:02:53,516::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314626 s, 32.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:02:54,080::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:02:55,528::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:02:55,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305819 s, 33.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:02:57,099::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:02:57,616::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:02:57,709::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.050279 s, 20.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:02:58,733::task::595::TaskManager.Task::(_updateState) Task=`9900c766-0337-4821-9cac-f4034ec2800c`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:02:58,734::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:02:58,734::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000669046', 'lastCheck': '7.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:02:58,734::task::1185::TaskManager.Task::(prepare) Task=`9900c766-0337-4821-9cac-f4034ec2800c`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000669046', 'lastCheck': '7.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:02:58,734::task::595::TaskManager.Task::(_updateState) Task=`9900c766-0337-4821-9cac-f4034ec2800c`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:02:58,734::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:02:58,735::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:02:58,735::task::990::TaskManager.Task::(_decref) Task=`9900c766-0337-4821-9cac-f4034ec2800c`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:02:59,722::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:02:59,803::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0370469 s, 27.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:03:00,117::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:03:01,093::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:03:01,102::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00074035 s, 5.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:03:01,624::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:03:01,625::task::595::TaskManager.Task::(_updateState) Task=`c8e4ff6d-2f78-4343-82f3-815a4ffec8e9`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:03:01,625::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:03:01,625::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:03:01,626::task::1185::TaskManager.Task::(prepare) Task=`c8e4ff6d-2f78-4343-82f3-815a4ffec8e9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:03:01,626::task::595::TaskManager.Task::(_updateState) Task=`c8e4ff6d-2f78-4343-82f3-815a4ffec8e9`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:03:01,626::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:03:01,626::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:03:01,626::task::990::TaskManager.Task::(_decref) Task=`c8e4ff6d-2f78-4343-82f3-815a4ffec8e9`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:03:01,631::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:03:01,631::task::595::TaskManager.Task::(_updateState) Task=`21bf4924-3720-464f-8ba4-059a6a9e91e3`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:03:01,632::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:03:01,632::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b8ed32b3-1d68-40c8-886d-af1111445f53`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:03:01,632::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:03:01,632::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:03:01,633::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b8ed32b3-1d68-40c8-886d-af1111445f53`::Granted request >Thread-338::DEBUG::2014-05-13 11:03:01,633::task::827::TaskManager.Task::(resourceAcquired) Task=`21bf4924-3720-464f-8ba4-059a6a9e91e3`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:03:01,633::task::990::TaskManager.Task::(_decref) Task=`21bf4924-3720-464f-8ba4-059a6a9e91e3`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:03:01,634::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:03:01,634::task::1185::TaskManager.Task::(prepare) Task=`21bf4924-3720-464f-8ba4-059a6a9e91e3`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:03:01,634::task::595::TaskManager.Task::(_updateState) Task=`21bf4924-3720-464f-8ba4-059a6a9e91e3`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:03:01,634::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:03:01,635::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:03:01,635::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:03:01,635::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:03:01,635::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:03:01,635::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:03:01,636::task::990::TaskManager.Task::(_decref) Task=`21bf4924-3720-464f-8ba4-059a6a9e91e3`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:03:01,815::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:03:01,897::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037294 s, 27.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:03:03,135::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:03:03,910::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:03:03,986::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314393 s, 32.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:03:05,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:03:06,081::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382163 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:03:06,154::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:03:08,094::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:03:08,171::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0333562 s, 30.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:03:09,171::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:03:10,184::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:03:10,259::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313919 s, 32.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:03:11,109::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:03:11,117::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00071655 s, 5.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:03:11,656::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:03:11,656::task::595::TaskManager.Task::(_updateState) Task=`e9808072-9068-4e89-8a11-5a18018e0243`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:03:11,657::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:03:11,657::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:03:11,657::task::1185::TaskManager.Task::(prepare) Task=`e9808072-9068-4e89-8a11-5a18018e0243`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:03:11,657::task::595::TaskManager.Task::(_updateState) Task=`e9808072-9068-4e89-8a11-5a18018e0243`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:03:11,657::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:03:11,658::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:03:11,658::task::990::TaskManager.Task::(_decref) Task=`e9808072-9068-4e89-8a11-5a18018e0243`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:03:11,663::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:03:11,664::task::595::TaskManager.Task::(_updateState) Task=`1bac5236-fdec-4777-b39c-1345cb7f3ed9`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:03:11,664::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:03:11,664::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c921158a-c3a3-4067-84ac-f3addaf10143`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:03:11,664::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:03:11,665::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:03:11,665::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c921158a-c3a3-4067-84ac-f3addaf10143`::Granted request >Thread-338::DEBUG::2014-05-13 11:03:11,665::task::827::TaskManager.Task::(resourceAcquired) Task=`1bac5236-fdec-4777-b39c-1345cb7f3ed9`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:03:11,666::task::990::TaskManager.Task::(_decref) Task=`1bac5236-fdec-4777-b39c-1345cb7f3ed9`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:03:11,666::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:03:11,666::task::1185::TaskManager.Task::(prepare) Task=`1bac5236-fdec-4777-b39c-1345cb7f3ed9`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:03:11,667::task::595::TaskManager.Task::(_updateState) Task=`1bac5236-fdec-4777-b39c-1345cb7f3ed9`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:03:11,667::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:03:11,667::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:03:11,667::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:03:11,667::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:03:11,668::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:03:11,668::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:03:11,668::task::990::TaskManager.Task::(_decref) Task=`1bac5236-fdec-4777-b39c-1345cb7f3ed9`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:03:12,190::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:03:12,272::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:03:12,679::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.364457 s, 2.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:03:13,878::task::595::TaskManager.Task::(_updateState) Task=`fdd006c8-faad-4be9-9c8a-3db40191240e`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:03:13,879::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:03:13,879::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00071655', 'lastCheck': '2.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:03:13,879::task::1185::TaskManager.Task::(prepare) Task=`fdd006c8-faad-4be9-9c8a-3db40191240e`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00071655', 'lastCheck': '2.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:03:13,879::task::595::TaskManager.Task::(_updateState) Task=`fdd006c8-faad-4be9-9c8a-3db40191240e`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:03:13,879::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:03:13,880::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:03:13,880::task::990::TaskManager.Task::(_decref) Task=`fdd006c8-faad-4be9-9c8a-3db40191240e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:03:14,692::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:03:14,774::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0372079 s, 27.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:03:15,208::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:03:16,787::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:03:16,866::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358624 s, 28.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:03:18,232::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:03:18,879::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:03:18,953::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0300971 s, 34.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:03:20,965::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:03:21,045::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361557 s, 28.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:03:21,125::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:03:21,133::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000594785 s, 6.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:03:21,252::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:03:21,689::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:03:21,689::task::595::TaskManager.Task::(_updateState) Task=`8a96e250-c6b8-4e78-95a0-76fa2755b129`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:03:21,689::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:03:21,690::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:03:21,690::task::1185::TaskManager.Task::(prepare) Task=`8a96e250-c6b8-4e78-95a0-76fa2755b129`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:03:21,690::task::595::TaskManager.Task::(_updateState) Task=`8a96e250-c6b8-4e78-95a0-76fa2755b129`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:03:21,690::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:03:21,690::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:03:21,690::task::990::TaskManager.Task::(_decref) Task=`8a96e250-c6b8-4e78-95a0-76fa2755b129`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:03:21,695::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:03:21,695::task::595::TaskManager.Task::(_updateState) Task=`cb68c47c-541e-4df2-a9a0-4d2fa0ac9d58`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:03:21,696::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:03:21,696::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2571cfad-d240-409d-8f94-2aa6cdb4bc75`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:03:21,696::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:03:21,696::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:03:21,696::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2571cfad-d240-409d-8f94-2aa6cdb4bc75`::Granted request >Thread-338::DEBUG::2014-05-13 11:03:21,697::task::827::TaskManager.Task::(resourceAcquired) Task=`cb68c47c-541e-4df2-a9a0-4d2fa0ac9d58`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:03:21,697::task::990::TaskManager.Task::(_decref) Task=`cb68c47c-541e-4df2-a9a0-4d2fa0ac9d58`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:03:21,698::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:03:21,698::task::1185::TaskManager.Task::(prepare) Task=`cb68c47c-541e-4df2-a9a0-4d2fa0ac9d58`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:03:21,698::task::595::TaskManager.Task::(_updateState) Task=`cb68c47c-541e-4df2-a9a0-4d2fa0ac9d58`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:03:21,698::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:03:21,698::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:03:21,698::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:03:21,699::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:03:21,699::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:03:21,699::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:03:21,699::task::990::TaskManager.Task::(_decref) Task=`cb68c47c-541e-4df2-a9a0-4d2fa0ac9d58`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:03:23,058::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:03:23,132::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307429 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:03:24,271::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:03:25,145::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:03:25,508::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.320319 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:03:27,290::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:03:27,521::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:03:27,599::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.034734 s, 29.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:03:29,020::task::595::TaskManager.Task::(_updateState) Task=`8c2c0a95-7610-4869-b13b-e42086efc564`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:03:29,020::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:03:29,020::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000594785', 'lastCheck': '7.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:03:29,020::task::1185::TaskManager.Task::(prepare) Task=`8c2c0a95-7610-4869-b13b-e42086efc564`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000594785', 'lastCheck': '7.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:03:29,020::task::595::TaskManager.Task::(_updateState) Task=`8c2c0a95-7610-4869-b13b-e42086efc564`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:03:29,021::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:03:29,021::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:03:29,021::task::990::TaskManager.Task::(_decref) Task=`8c2c0a95-7610-4869-b13b-e42086efc564`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:03:29,612::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:03:29,688::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318606 s, 32.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:03:30,314::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:03:31,140::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:03:31,149::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000790491 s, 5.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:03:31,701::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 11:03:31,721::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [30b3ccf] >Thread-66::DEBUG::2014-05-13 11:03:31,723::task::595::TaskManager.Task::(_updateState) Task=`e4e4a0e6-f311-4c3d-910a-60690393f8ee`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:03:31,724::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:03:31,726::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:03:31,726::task::1185::TaskManager.Task::(prepare) Task=`e4e4a0e6-f311-4c3d-910a-60690393f8ee`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:03:31,727::task::595::TaskManager.Task::(_updateState) Task=`e4e4a0e6-f311-4c3d-910a-60690393f8ee`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:03:31,728::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:03:31,729::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:03:31,730::task::990::TaskManager.Task::(_decref) Task=`e4e4a0e6-f311-4c3d-910a-60690393f8ee`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:03:31,738::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [30b3ccf] >Thread-338::DEBUG::2014-05-13 11:03:31,739::task::595::TaskManager.Task::(_updateState) Task=`9470c1e3-6b8c-4c52-9474-f0861f96eafa`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:03:31,740::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:03:31,742::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`16b74e08-1b8d-4654-8098-d59ac225d51a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:03:31,743::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:03:31,744::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:03:31,745::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`16b74e08-1b8d-4654-8098-d59ac225d51a`::Granted request >Thread-338::DEBUG::2014-05-13 11:03:31,747::task::827::TaskManager.Task::(resourceAcquired) Task=`9470c1e3-6b8c-4c52-9474-f0861f96eafa`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:03:31,748::task::990::TaskManager.Task::(_decref) Task=`9470c1e3-6b8c-4c52-9474-f0861f96eafa`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:03:31,753::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:03:31,754::task::1185::TaskManager.Task::(prepare) Task=`9470c1e3-6b8c-4c52-9474-f0861f96eafa`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:03:31,754::task::595::TaskManager.Task::(_updateState) Task=`9470c1e3-6b8c-4c52-9474-f0861f96eafa`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:03:31,755::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:03:31,756::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:03:31,758::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:03:31,759::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:03:31,760::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:03:31,761::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:03:31,762::task::990::TaskManager.Task::(_decref) Task=`9470c1e3-6b8c-4c52-9474-f0861f96eafa`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:03:31,818::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.074261 s, 13.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:03:33,338::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:03:33,831::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:03:33,909::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.033653 s, 30.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:03:35,922::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:03:36,008::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363273 s, 28.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:03:36,358::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:03:38,023::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:03:38,380::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.313271 s, 3.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:03:39,386::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:03:40,429::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:03:40,508::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0345952 s, 29.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:03:41,156::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:03:41,164::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000712234 s, 5.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:03:41,783::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6849800f] >Thread-66::DEBUG::2014-05-13 11:03:41,784::task::595::TaskManager.Task::(_updateState) Task=`5a9a98c1-404d-4701-a1ff-fccca62a12dc`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:03:41,784::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:03:41,784::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:03:41,784::task::1185::TaskManager.Task::(prepare) Task=`5a9a98c1-404d-4701-a1ff-fccca62a12dc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:03:41,785::task::595::TaskManager.Task::(_updateState) Task=`5a9a98c1-404d-4701-a1ff-fccca62a12dc`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:03:41,785::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:03:41,785::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:03:41,785::task::990::TaskManager.Task::(_decref) Task=`5a9a98c1-404d-4701-a1ff-fccca62a12dc`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:03:41,790::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6849800f] >Thread-338::DEBUG::2014-05-13 11:03:41,790::task::595::TaskManager.Task::(_updateState) Task=`0500cc2a-e5fc-487d-9193-4afde03c8eaa`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:03:41,790::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:03:41,791::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ac22ca14-10fe-429c-b1d1-aa2fe6a4e699`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:03:41,791::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:03:41,791::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:03:41,791::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ac22ca14-10fe-429c-b1d1-aa2fe6a4e699`::Granted request >Thread-338::DEBUG::2014-05-13 11:03:41,792::task::827::TaskManager.Task::(resourceAcquired) Task=`0500cc2a-e5fc-487d-9193-4afde03c8eaa`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:03:41,792::task::990::TaskManager.Task::(_decref) Task=`0500cc2a-e5fc-487d-9193-4afde03c8eaa`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:03:41,793::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:03:41,793::task::1185::TaskManager.Task::(prepare) Task=`0500cc2a-e5fc-487d-9193-4afde03c8eaa`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:03:41,793::task::595::TaskManager.Task::(_updateState) Task=`0500cc2a-e5fc-487d-9193-4afde03c8eaa`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:03:41,793::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:03:41,793::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:03:41,794::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:03:41,794::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:03:41,794::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:03:41,794::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:03:41,794::task::990::TaskManager.Task::(_decref) Task=`0500cc2a-e5fc-487d-9193-4afde03c8eaa`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:03:42,415::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:03:42,522::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:03:42,608::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0437186 s, 23.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:03:44,152::task::595::TaskManager.Task::(_updateState) Task=`540df1ad-e70f-4c45-8387-6ee064a1e2db`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:03:44,152::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:03:44,152::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000712234', 'lastCheck': '3.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:03:44,153::task::1185::TaskManager.Task::(prepare) Task=`540df1ad-e70f-4c45-8387-6ee064a1e2db`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000712234', 'lastCheck': '3.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:03:44,153::task::595::TaskManager.Task::(_updateState) Task=`540df1ad-e70f-4c45-8387-6ee064a1e2db`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:03:44,153::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:03:44,153::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:03:44,153::task::990::TaskManager.Task::(_decref) Task=`540df1ad-e70f-4c45-8387-6ee064a1e2db`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:03:44,620::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:03:44,707::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305359 s, 33.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:03:45,436::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:03:46,720::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:03:46,802::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0378947 s, 27.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:03:48,456::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:03:48,814::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:03:48,893::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0338788 s, 30.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:03:50,907::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:03:50,998::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0480661 s, 21.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:03:51,172::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:03:51,180::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000690694 s, 5.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:03:51,476::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:03:51,816::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1c80df6] >Thread-66::DEBUG::2014-05-13 11:03:51,817::task::595::TaskManager.Task::(_updateState) Task=`af68438d-8197-4c60-947b-e3181d987849`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:03:51,817::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:03:51,817::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:03:51,817::task::1185::TaskManager.Task::(prepare) Task=`af68438d-8197-4c60-947b-e3181d987849`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:03:51,817::task::595::TaskManager.Task::(_updateState) Task=`af68438d-8197-4c60-947b-e3181d987849`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:03:51,818::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:03:51,818::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:03:51,818::task::990::TaskManager.Task::(_decref) Task=`af68438d-8197-4c60-947b-e3181d987849`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:03:51,823::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1c80df6] >Thread-338::DEBUG::2014-05-13 11:03:51,824::task::595::TaskManager.Task::(_updateState) Task=`6e180c3e-25ce-4625-94eb-dc789dfb8e45`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:03:51,824::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:03:51,824::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`56f4927c-f34d-451e-b464-b0036aac1943`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:03:51,824::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:03:51,825::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:03:51,825::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`56f4927c-f34d-451e-b464-b0036aac1943`::Granted request >Thread-338::DEBUG::2014-05-13 11:03:51,825::task::827::TaskManager.Task::(resourceAcquired) Task=`6e180c3e-25ce-4625-94eb-dc789dfb8e45`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:03:51,825::task::990::TaskManager.Task::(_decref) Task=`6e180c3e-25ce-4625-94eb-dc789dfb8e45`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:03:51,826::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:03:51,826::task::1185::TaskManager.Task::(prepare) Task=`6e180c3e-25ce-4625-94eb-dc789dfb8e45`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:03:51,826::task::595::TaskManager.Task::(_updateState) Task=`6e180c3e-25ce-4625-94eb-dc789dfb8e45`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:03:51,826::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:03:51,827::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:03:51,827::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:03:51,827::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:03:51,827::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:03:51,827::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:03:51,827::task::990::TaskManager.Task::(_decref) Task=`6e180c3e-25ce-4625-94eb-dc789dfb8e45`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:03:53,011::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:03:53,086::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316684 s, 32.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:03:54,498::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:03:55,099::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:03:55,174::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312037 s, 32.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:03:57,187::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:03:57,269::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0367465 s, 27.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:03:57,521::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:03:59,282::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 11:03:59,290::task::595::TaskManager.Task::(_updateState) Task=`e1e29b71-a953-46ec-9ebf-bc157b16becb`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:03:59,290::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:03:59,291::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000690694', 'lastCheck': '8.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:03:59,291::task::1185::TaskManager.Task::(prepare) Task=`e1e29b71-a953-46ec-9ebf-bc157b16becb`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000690694', 'lastCheck': '8.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:03:59,291::task::595::TaskManager.Task::(_updateState) Task=`e1e29b71-a953-46ec-9ebf-bc157b16becb`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:03:59,291::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:03:59,291::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:03:59,292::task::990::TaskManager.Task::(_decref) Task=`e1e29b71-a953-46ec-9ebf-bc157b16becb`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:03:59,358::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318665 s, 32.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:04:00,538::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:04:01,188::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:04:01,196::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000633497 s, 6.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:04:01,371::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:04:01,450::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357116 s, 28.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:04:01,853::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [12ac301a] >Thread-66::DEBUG::2014-05-13 11:04:01,853::task::595::TaskManager.Task::(_updateState) Task=`e0a865e3-13c0-46d8-8447-44790942731e`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:04:01,853::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:04:01,854::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:04:01,854::task::1185::TaskManager.Task::(prepare) Task=`e0a865e3-13c0-46d8-8447-44790942731e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:04:01,854::task::595::TaskManager.Task::(_updateState) Task=`e0a865e3-13c0-46d8-8447-44790942731e`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:04:01,854::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:04:01,854::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:04:01,854::task::990::TaskManager.Task::(_decref) Task=`e0a865e3-13c0-46d8-8447-44790942731e`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:04:01,859::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [12ac301a] >Thread-338::DEBUG::2014-05-13 11:04:01,860::task::595::TaskManager.Task::(_updateState) Task=`b4b3412a-4de3-41f7-b6d4-84d4c30053b0`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:04:01,860::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:04:01,860::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`97c53150-0598-4a29-8c7a-b57588e5abd9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:04:01,861::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:04:01,861::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:04:01,861::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`97c53150-0598-4a29-8c7a-b57588e5abd9`::Granted request >Thread-338::DEBUG::2014-05-13 11:04:01,861::task::827::TaskManager.Task::(resourceAcquired) Task=`b4b3412a-4de3-41f7-b6d4-84d4c30053b0`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:04:01,862::task::990::TaskManager.Task::(_decref) Task=`b4b3412a-4de3-41f7-b6d4-84d4c30053b0`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:04:01,862::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:04:01,863::task::1185::TaskManager.Task::(prepare) Task=`b4b3412a-4de3-41f7-b6d4-84d4c30053b0`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:04:01,863::task::595::TaskManager.Task::(_updateState) Task=`b4b3412a-4de3-41f7-b6d4-84d4c30053b0`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:04:01,863::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:04:01,863::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:04:01,863::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:04:01,864::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:04:01,864::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:04:01,864::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:04:01,864::task::990::TaskManager.Task::(_decref) Task=`b4b3412a-4de3-41f7-b6d4-84d4c30053b0`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:04:03,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:04:03,557::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:04:03,561::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0505616 s, 20.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:04:05,574::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:04:05,648::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0304955 s, 33.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:04:06,574::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:04:07,660::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:04:07,735::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315822 s, 32.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:04:09,593::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:04:09,748::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:04:09,826::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352579 s, 29.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:04:11,203::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:04:11,212::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000757786 s, 5.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:04:11,839::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 11:04:11,884::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:04:11,886::task::595::TaskManager.Task::(_updateState) Task=`bed67ad3-ba9a-4e85-ade4-7e38f6ee7bf9`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:04:11,887::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:04:11,889::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:04:11,890::task::1185::TaskManager.Task::(prepare) Task=`bed67ad3-ba9a-4e85-ade4-7e38f6ee7bf9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:04:11,891::task::595::TaskManager.Task::(_updateState) Task=`bed67ad3-ba9a-4e85-ade4-7e38f6ee7bf9`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:04:11,892::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:04:11,893::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:04:11,894::task::990::TaskManager.Task::(_decref) Task=`bed67ad3-ba9a-4e85-ade4-7e38f6ee7bf9`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:04:11,902::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:04:11,903::task::595::TaskManager.Task::(_updateState) Task=`6f3c4a0a-bfce-404b-a690-68451acab5e5`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:04:11,905::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:04:11,907::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5e3fd2c0-4ba8-4584-aeef-4fef9d4bd550`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:04:11,908::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:04:11,909::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:04:11,910::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5e3fd2c0-4ba8-4584-aeef-4fef9d4bd550`::Granted request >Thread-338::DEBUG::2014-05-13 11:04:11,912::task::827::TaskManager.Task::(resourceAcquired) Task=`6f3c4a0a-bfce-404b-a690-68451acab5e5`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:04:11,913::task::990::TaskManager.Task::(_decref) Task=`6f3c4a0a-bfce-404b-a690-68451acab5e5`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:04:11,917::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:04:11,918::task::1185::TaskManager.Task::(prepare) Task=`6f3c4a0a-bfce-404b-a690-68451acab5e5`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:04:11,919::task::595::TaskManager.Task::(_updateState) Task=`6f3c4a0a-bfce-404b-a690-68451acab5e5`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:04:11,920::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:04:11,921::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:04:11,922::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:04:11,924::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:04:11,925::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:04:11,925::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:04:11,927::task::990::TaskManager.Task::(_decref) Task=`6f3c4a0a-bfce-404b-a690-68451acab5e5`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:04:11,959::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0476289 s, 21.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:04:12,621::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:04:13,972::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:04:14,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0326621 s, 31.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:04:14,456::task::595::TaskManager.Task::(_updateState) Task=`78a562e3-2f4e-480c-9170-f8355aabef61`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:04:14,456::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:04:14,457::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000757786', 'lastCheck': '3.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:04:14,457::task::1185::TaskManager.Task::(prepare) Task=`78a562e3-2f4e-480c-9170-f8355aabef61`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000757786', 'lastCheck': '3.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:04:14,457::task::595::TaskManager.Task::(_updateState) Task=`78a562e3-2f4e-480c-9170-f8355aabef61`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:04:14,457::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:04:14,457::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:04:14,458::task::990::TaskManager.Task::(_decref) Task=`78a562e3-2f4e-480c-9170-f8355aabef61`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:04:15,698::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:04:16,061::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:04:16,137::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0330857 s, 30.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:04:18,150::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:04:18,560::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.367829 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:04:18,717::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:04:20,574::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:04:20,648::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0300547 s, 34.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:04:21,219::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:04:21,227::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00060855 s, 6.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:04:21,737::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:04:21,954::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:04:21,954::task::595::TaskManager.Task::(_updateState) Task=`1f2d28de-0a37-424d-bfc7-3c0bc528f68d`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:04:21,954::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:04:21,955::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:04:21,955::task::1185::TaskManager.Task::(prepare) Task=`1f2d28de-0a37-424d-bfc7-3c0bc528f68d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:04:21,955::task::595::TaskManager.Task::(_updateState) Task=`1f2d28de-0a37-424d-bfc7-3c0bc528f68d`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:04:21,955::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:04:21,955::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:04:21,955::task::990::TaskManager.Task::(_decref) Task=`1f2d28de-0a37-424d-bfc7-3c0bc528f68d`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:04:21,960::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:04:21,960::task::595::TaskManager.Task::(_updateState) Task=`1f8e0676-8eba-4ccc-bad3-644467edb400`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:04:21,961::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:04:21,961::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d9c96d88-c892-4c64-b3e1-67c3cb4eb68c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:04:21,961::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:04:21,961::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:04:21,962::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d9c96d88-c892-4c64-b3e1-67c3cb4eb68c`::Granted request >Thread-338::DEBUG::2014-05-13 11:04:21,962::task::827::TaskManager.Task::(resourceAcquired) Task=`1f8e0676-8eba-4ccc-bad3-644467edb400`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:04:21,962::task::990::TaskManager.Task::(_decref) Task=`1f8e0676-8eba-4ccc-bad3-644467edb400`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:04:21,963::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:04:21,963::task::1185::TaskManager.Task::(prepare) Task=`1f8e0676-8eba-4ccc-bad3-644467edb400`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:04:21,963::task::595::TaskManager.Task::(_updateState) Task=`1f8e0676-8eba-4ccc-bad3-644467edb400`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:04:21,963::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:04:21,963::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:04:21,964::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:04:21,964::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:04:21,964::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:04:21,964::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:04:21,964::task::990::TaskManager.Task::(_decref) Task=`1f8e0676-8eba-4ccc-bad3-644467edb400`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:04:22,660::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:04:22,737::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0332606 s, 30.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:04:24,750::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:04:24,767::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:04:24,836::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311352 s, 32.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:04:26,849::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:04:26,930::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0371489 s, 27.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:04:27,785::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:04:28,943::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:04:29,017::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0302867 s, 33.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:04:29,602::task::595::TaskManager.Task::(_updateState) Task=`c8c2d1f9-41f5-46d2-aaf1-3f0a38e50f38`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:04:29,602::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:04:29,603::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00060855', 'lastCheck': '8.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:04:29,603::task::1185::TaskManager.Task::(prepare) Task=`c8c2d1f9-41f5-46d2-aaf1-3f0a38e50f38`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00060855', 'lastCheck': '8.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:04:29,603::task::595::TaskManager.Task::(_updateState) Task=`c8c2d1f9-41f5-46d2-aaf1-3f0a38e50f38`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:04:29,603::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:04:29,603::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:04:29,604::task::990::TaskManager.Task::(_decref) Task=`c8c2d1f9-41f5-46d2-aaf1-3f0a38e50f38`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:04:30,805::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:04:31,030::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:04:31,119::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0445674 s, 23.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:04:31,236::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:04:31,246::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000766232 s, 5.3 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:04:31,994::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:04:31,995::task::595::TaskManager.Task::(_updateState) Task=`d72570d4-96ad-4360-bc04-b0bce258dd80`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:04:31,995::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:04:31,995::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:04:31,995::task::1185::TaskManager.Task::(prepare) Task=`d72570d4-96ad-4360-bc04-b0bce258dd80`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:04:31,996::task::595::TaskManager.Task::(_updateState) Task=`d72570d4-96ad-4360-bc04-b0bce258dd80`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:04:31,996::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:04:31,996::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:04:31,996::task::990::TaskManager.Task::(_decref) Task=`d72570d4-96ad-4360-bc04-b0bce258dd80`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:04:32,001::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:04:32,001::task::595::TaskManager.Task::(_updateState) Task=`86389ec1-775f-4862-813f-d9471f267b0c`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:04:32,001::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:04:32,002::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a9452ff5-12e6-4008-baf1-efc53f8a4bd9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:04:32,002::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:04:32,002::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:04:32,002::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a9452ff5-12e6-4008-baf1-efc53f8a4bd9`::Granted request >Thread-338::DEBUG::2014-05-13 11:04:32,003::task::827::TaskManager.Task::(resourceAcquired) Task=`86389ec1-775f-4862-813f-d9471f267b0c`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:04:32,003::task::990::TaskManager.Task::(_decref) Task=`86389ec1-775f-4862-813f-d9471f267b0c`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:04:32,004::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:04:32,004::task::1185::TaskManager.Task::(prepare) Task=`86389ec1-775f-4862-813f-d9471f267b0c`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:04:32,004::task::595::TaskManager.Task::(_updateState) Task=`86389ec1-775f-4862-813f-d9471f267b0c`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:04:32,004::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:04:32,004::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:04:32,005::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:04:32,005::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:04:32,005::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:04:32,005::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:04:32,005::task::990::TaskManager.Task::(_decref) Task=`86389ec1-775f-4862-813f-d9471f267b0c`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:04:33,132::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:04:33,212::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357667 s, 28.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:04:33,823::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:04:35,225::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:04:35,298::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0300226 s, 34.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:04:36,843::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:04:37,311::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:04:37,382::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0295823 s, 34.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:04:39,395::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:04:39,474::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352306 s, 29.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:04:39,863::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:04:41,253::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:04:41,261::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000673146 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:04:41,488::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:04:41,566::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0351475 s, 29.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:04:42,025::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:04:42,025::task::595::TaskManager.Task::(_updateState) Task=`7ddfa26f-9341-4282-bcf5-f348589d2834`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:04:42,026::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:04:42,026::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:04:42,026::task::1185::TaskManager.Task::(prepare) Task=`7ddfa26f-9341-4282-bcf5-f348589d2834`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:04:42,026::task::595::TaskManager.Task::(_updateState) Task=`7ddfa26f-9341-4282-bcf5-f348589d2834`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:04:42,026::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:04:42,027::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:04:42,027::task::990::TaskManager.Task::(_decref) Task=`7ddfa26f-9341-4282-bcf5-f348589d2834`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:04:42,031::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:04:42,032::task::595::TaskManager.Task::(_updateState) Task=`f8b3b686-0692-44f5-9470-eb974baa7835`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:04:42,032::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:04:42,032::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b780ddbf-ec09-483f-b9d4-2e41bae86480`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:04:42,033::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:04:42,033::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:04:42,033::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b780ddbf-ec09-483f-b9d4-2e41bae86480`::Granted request >Thread-338::DEBUG::2014-05-13 11:04:42,033::task::827::TaskManager.Task::(resourceAcquired) Task=`f8b3b686-0692-44f5-9470-eb974baa7835`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:04:42,034::task::990::TaskManager.Task::(_decref) Task=`f8b3b686-0692-44f5-9470-eb974baa7835`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:04:42,034::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:04:42,035::task::1185::TaskManager.Task::(prepare) Task=`f8b3b686-0692-44f5-9470-eb974baa7835`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:04:42,035::task::595::TaskManager.Task::(_updateState) Task=`f8b3b686-0692-44f5-9470-eb974baa7835`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:04:42,035::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:04:42,035::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:04:42,035::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:04:42,036::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:04:42,036::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:04:42,036::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:04:42,036::task::990::TaskManager.Task::(_decref) Task=`f8b3b686-0692-44f5-9470-eb974baa7835`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:04:42,888::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:04:43,578::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:04:43,670::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0469175 s, 21.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:04:44,735::task::595::TaskManager.Task::(_updateState) Task=`14fa432e-1c56-47f4-8a25-3b64482baa5a`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:04:44,735::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:04:44,735::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000673146', 'lastCheck': '3.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:04:44,736::task::1185::TaskManager.Task::(prepare) Task=`14fa432e-1c56-47f4-8a25-3b64482baa5a`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000673146', 'lastCheck': '3.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:04:44,736::task::595::TaskManager.Task::(_updateState) Task=`14fa432e-1c56-47f4-8a25-3b64482baa5a`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:04:44,736::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:04:44,736::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:04:44,736::task::990::TaskManager.Task::(_decref) Task=`14fa432e-1c56-47f4-8a25-3b64482baa5a`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:04:45,682::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:04:45,755::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0302369 s, 33.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:04:45,908::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:04:47,768::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:04:47,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0351984 s, 29.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:04:48,925::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:04:49,864::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:04:49,938::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314181 s, 32.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:04:51,265::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 41830dad-f012-4b4d-b636-863f66d5ca4b >Thread-322::DEBUG::2014-05-13 11:04:51,266::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-322::DEBUG::2014-05-13 11:04:51,268::blockSD::336::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:04:51,276::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000600064 s, 3.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:04:51,277::misc::287::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000600064 s, 3.4 MB/s'], size: 2048 >Thread-322::DEBUG::2014-05-13 11:04:51,277::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-322::WARNING::2014-05-13 11:04:51,277::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-322::DEBUG::2014-05-13 11:04:51,277::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-322::DEBUG::2014-05-13 11:04:51,278::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:04:51,278::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:04:51,278::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:04:51,278::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:04:51,279::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:04:51,280::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 11:04:51,514::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:04:51,516::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:04:51,517::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'LOCKPOLICY=', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', 'DESCRIPTION=ISCSI', 'LOGBLKSIZE=512', 'TYPE=ISCSI', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'PHYBLKSIZE=512', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', 'CLASS=Data', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'POOL_DESCRIPTION=DC_2', 'POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'MASTER_VERSION=1', 'ROLE=Master', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', '_SHA_CKSUM=28bb2d7cc60e0d779eb69fe93d050c16c769f069', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0'] >Thread-322::INFO::2014-05-13 11:04:51,518::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_imageNS already registered >Thread-322::INFO::2014-05-13 11:04:51,518::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_volumeNS already registered >Thread-322::INFO::2014-05-13 11:04:51,518::blockSD::452::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_lvmActivationNS already registered >Thread-322::DEBUG::2014-05-13 11:04:51,519::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 11:04:51,709::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:04:51,712::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:04:51,721::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000647188 s, 6.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:04:51,948::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:04:51,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:04:52,029::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0354143 s, 28.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:04:52,055::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:04:52,056::task::595::TaskManager.Task::(_updateState) Task=`01080101-ed7b-4dec-8f89-a780e8aa8e2a`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:04:52,056::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:04:52,057::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:04:52,057::task::1185::TaskManager.Task::(prepare) Task=`01080101-ed7b-4dec-8f89-a780e8aa8e2a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:04:52,057::task::595::TaskManager.Task::(_updateState) Task=`01080101-ed7b-4dec-8f89-a780e8aa8e2a`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:04:52,057::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:04:52,057::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:04:52,057::task::990::TaskManager.Task::(_decref) Task=`01080101-ed7b-4dec-8f89-a780e8aa8e2a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:04:52,063::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:04:52,063::task::595::TaskManager.Task::(_updateState) Task=`a2539eec-337d-4ace-9c77-6d2076ed4efd`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:04:52,063::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:04:52,064::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`878a7316-99f6-4a71-880e-a2998c2535b1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:04:52,064::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:04:52,064::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:04:52,064::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`878a7316-99f6-4a71-880e-a2998c2535b1`::Granted request >Thread-338::DEBUG::2014-05-13 11:04:52,065::task::827::TaskManager.Task::(resourceAcquired) Task=`a2539eec-337d-4ace-9c77-6d2076ed4efd`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:04:52,065::task::990::TaskManager.Task::(_decref) Task=`a2539eec-337d-4ace-9c77-6d2076ed4efd`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:04:52,066::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:04:52,066::task::1185::TaskManager.Task::(prepare) Task=`a2539eec-337d-4ace-9c77-6d2076ed4efd`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:04:52,066::task::595::TaskManager.Task::(_updateState) Task=`a2539eec-337d-4ace-9c77-6d2076ed4efd`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:04:52,066::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:04:52,066::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:04:52,066::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:04:52,067::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:04:52,067::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:04:52,067::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:04:52,067::task::990::TaskManager.Task::(_decref) Task=`a2539eec-337d-4ace-9c77-6d2076ed4efd`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:04:54,042::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:04:54,121::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355222 s, 28.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:04:54,968::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:04:56,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:04:56,209::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311364 s, 32.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:04:57,987::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:04:58,222::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:04:58,580::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.315852 s, 3.2 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:04:59,868::task::595::TaskManager.Task::(_updateState) Task=`1d6726fd-8980-4ccf-9bff-bbf3740aaccd`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:04:59,869::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:04:59,869::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000647188', 'lastCheck': '8.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:04:59,869::task::1185::TaskManager.Task::(prepare) Task=`1d6726fd-8980-4ccf-9bff-bbf3740aaccd`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000647188', 'lastCheck': '8.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:04:59,869::task::595::TaskManager.Task::(_updateState) Task=`1d6726fd-8980-4ccf-9bff-bbf3740aaccd`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:04:59,869::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:04:59,870::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:04:59,870::task::990::TaskManager.Task::(_decref) Task=`1d6726fd-8980-4ccf-9bff-bbf3740aaccd`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:05:00,593::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:00,672::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0367285 s, 27.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:05:01,005::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:05:01,728::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:05:01,737::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000580422 s, 7.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:05:02,089::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:05:02,090::task::595::TaskManager.Task::(_updateState) Task=`96eb5ca4-522a-426a-a8ab-07919de8fe79`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:05:02,090::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:05:02,090::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:05:02,091::task::1185::TaskManager.Task::(prepare) Task=`96eb5ca4-522a-426a-a8ab-07919de8fe79`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:05:02,091::task::595::TaskManager.Task::(_updateState) Task=`96eb5ca4-522a-426a-a8ab-07919de8fe79`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:05:02,091::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:05:02,091::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:05:02,091::task::990::TaskManager.Task::(_decref) Task=`96eb5ca4-522a-426a-a8ab-07919de8fe79`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:05:02,097::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:05:02,097::task::595::TaskManager.Task::(_updateState) Task=`c8a384c9-7325-479e-8f81-3ac82bfcbdf2`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:05:02,097::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:05:02,098::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`74106709-3025-4a6e-91ef-a313ed6e43d1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:05:02,098::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:05:02,098::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:05:02,098::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`74106709-3025-4a6e-91ef-a313ed6e43d1`::Granted request >Thread-338::DEBUG::2014-05-13 11:05:02,099::task::827::TaskManager.Task::(resourceAcquired) Task=`c8a384c9-7325-479e-8f81-3ac82bfcbdf2`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:05:02,099::task::990::TaskManager.Task::(_decref) Task=`c8a384c9-7325-479e-8f81-3ac82bfcbdf2`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:05:02,100::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:05:02,100::task::1185::TaskManager.Task::(prepare) Task=`c8a384c9-7325-479e-8f81-3ac82bfcbdf2`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:05:02,100::task::595::TaskManager.Task::(_updateState) Task=`c8a384c9-7325-479e-8f81-3ac82bfcbdf2`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:05:02,100::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:05:02,100::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:05:02,101::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:05:02,101::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:05:02,101::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:05:02,101::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:05:02,101::task::990::TaskManager.Task::(_decref) Task=`c8a384c9-7325-479e-8f81-3ac82bfcbdf2`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:05:02,686::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:02,764::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0351181 s, 29.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:05:04,024::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:05:04,777::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:04,852::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031918 s, 32.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:05:06,865::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:06,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0371686 s, 27.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:05:07,042::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:05:08,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:09,047::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0392805 s, 26.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:05:10,062::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:05:11,060::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:11,147::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0447259 s, 22.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:05:11,744::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:05:11,752::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000716906 s, 5.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:05:12,118::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [72471f8f] >Thread-66::DEBUG::2014-05-13 11:05:12,119::task::595::TaskManager.Task::(_updateState) Task=`9e861a59-f8f4-44e4-b780-28804d2269be`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:05:12,119::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:05:12,119::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:05:12,119::task::1185::TaskManager.Task::(prepare) Task=`9e861a59-f8f4-44e4-b780-28804d2269be`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:05:12,119::task::595::TaskManager.Task::(_updateState) Task=`9e861a59-f8f4-44e4-b780-28804d2269be`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:05:12,120::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:05:12,120::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:05:12,120::task::990::TaskManager.Task::(_decref) Task=`9e861a59-f8f4-44e4-b780-28804d2269be`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:05:12,171::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [72471f8f] >Thread-338::DEBUG::2014-05-13 11:05:12,172::task::595::TaskManager.Task::(_updateState) Task=`42d13d02-dd3a-4722-aea4-0b9fae898306`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:05:12,172::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:05:12,172::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5a8f8c72-116d-41a2-a73c-42aefadcc629`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:05:12,172::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:05:12,173::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:05:12,173::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5a8f8c72-116d-41a2-a73c-42aefadcc629`::Granted request >Thread-338::DEBUG::2014-05-13 11:05:12,173::task::827::TaskManager.Task::(resourceAcquired) Task=`42d13d02-dd3a-4722-aea4-0b9fae898306`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:05:12,173::task::990::TaskManager.Task::(_decref) Task=`42d13d02-dd3a-4722-aea4-0b9fae898306`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:05:12,174::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:05:12,174::task::1185::TaskManager.Task::(prepare) Task=`42d13d02-dd3a-4722-aea4-0b9fae898306`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:05:12,174::task::595::TaskManager.Task::(_updateState) Task=`42d13d02-dd3a-4722-aea4-0b9fae898306`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:05:12,175::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:05:12,175::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:05:12,175::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:05:12,175::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:05:12,175::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:05:12,176::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:05:12,176::task::990::TaskManager.Task::(_decref) Task=`42d13d02-dd3a-4722-aea4-0b9fae898306`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:05:13,080::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:05:13,160::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:13,234::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309197 s, 33.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:05:15,013::task::595::TaskManager.Task::(_updateState) Task=`ac378d77-bd6f-463c-88ed-1e642e8c3857`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:05:15,013::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:05:15,013::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000716906', 'lastCheck': '3.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:05:15,013::task::1185::TaskManager.Task::(prepare) Task=`ac378d77-bd6f-463c-88ed-1e642e8c3857`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000716906', 'lastCheck': '3.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:05:15,014::task::595::TaskManager.Task::(_updateState) Task=`ac378d77-bd6f-463c-88ed-1e642e8c3857`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:05:15,014::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:05:15,014::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:05:15,014::task::990::TaskManager.Task::(_decref) Task=`ac378d77-bd6f-463c-88ed-1e642e8c3857`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:05:15,247::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:15,327::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361138 s, 28.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:05:16,098::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:05:17,340::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:17,419::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359034 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:05:19,121::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:05:19,432::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:19,506::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309149 s, 33.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:05:21,519::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:21,598::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357917 s, 28.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:05:21,759::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:05:21,768::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000714641 s, 5.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:05:22,144::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:05:22,196::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:05:22,197::task::595::TaskManager.Task::(_updateState) Task=`cc94e486-6bd0-465f-861e-b3164e917e65`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:05:22,197::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:05:22,197::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:05:22,198::task::1185::TaskManager.Task::(prepare) Task=`cc94e486-6bd0-465f-861e-b3164e917e65`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:05:22,198::task::595::TaskManager.Task::(_updateState) Task=`cc94e486-6bd0-465f-861e-b3164e917e65`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:05:22,198::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:05:22,198::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:05:22,198::task::990::TaskManager.Task::(_decref) Task=`cc94e486-6bd0-465f-861e-b3164e917e65`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:05:22,203::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:05:22,203::task::595::TaskManager.Task::(_updateState) Task=`fed064f8-b940-4fd6-ae7b-323bc684cfcb`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:05:22,203::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:05:22,204::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`cf3368ee-f174-4f4e-92db-62073b3ac3c2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:05:22,204::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:05:22,204::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:05:22,205::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`cf3368ee-f174-4f4e-92db-62073b3ac3c2`::Granted request >Thread-338::DEBUG::2014-05-13 11:05:22,205::task::827::TaskManager.Task::(resourceAcquired) Task=`fed064f8-b940-4fd6-ae7b-323bc684cfcb`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:05:22,205::task::990::TaskManager.Task::(_decref) Task=`fed064f8-b940-4fd6-ae7b-323bc684cfcb`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:05:22,206::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:05:22,206::task::1185::TaskManager.Task::(prepare) Task=`fed064f8-b940-4fd6-ae7b-323bc684cfcb`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:05:22,206::task::595::TaskManager.Task::(_updateState) Task=`fed064f8-b940-4fd6-ae7b-323bc684cfcb`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:05:22,206::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:05:22,206::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:05:22,207::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:05:22,207::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:05:22,207::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:05:22,207::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:05:22,207::task::990::TaskManager.Task::(_decref) Task=`fed064f8-b940-4fd6-ae7b-323bc684cfcb`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:05:23,611::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:23,702::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0484603 s, 21.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:05:25,163::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:05:25,715::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:25,793::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352464 s, 29.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:05:27,806::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:27,880::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0304323 s, 33.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:05:28,181::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:05:29,892::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:29,972::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0376667 s, 27.2 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:05:30,156::task::595::TaskManager.Task::(_updateState) Task=`4375670c-8a05-48af-ae66-37a39ad9f677`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:05:30,156::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:05:30,157::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000714641', 'lastCheck': '8.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:05:30,157::task::1185::TaskManager.Task::(prepare) Task=`4375670c-8a05-48af-ae66-37a39ad9f677`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000714641', 'lastCheck': '8.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:05:30,157::task::595::TaskManager.Task::(_updateState) Task=`4375670c-8a05-48af-ae66-37a39ad9f677`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:05:30,157::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:05:30,157::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:05:30,157::task::990::TaskManager.Task::(_decref) Task=`4375670c-8a05-48af-ae66-37a39ad9f677`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:05:31,198::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:05:31,775::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:05:31,784::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000883619 s, 4.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:05:31,985::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:32,064::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.03582 s, 28.6 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:05:32,224::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:05:32,225::task::595::TaskManager.Task::(_updateState) Task=`44434a25-b70b-4842-8b0b-e470e8941ecb`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:05:32,225::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:05:32,225::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:05:32,225::task::1185::TaskManager.Task::(prepare) Task=`44434a25-b70b-4842-8b0b-e470e8941ecb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:05:32,226::task::595::TaskManager.Task::(_updateState) Task=`44434a25-b70b-4842-8b0b-e470e8941ecb`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:05:32,226::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:05:32,226::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:05:32,226::task::990::TaskManager.Task::(_decref) Task=`44434a25-b70b-4842-8b0b-e470e8941ecb`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:05:32,230::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:05:32,231::task::595::TaskManager.Task::(_updateState) Task=`f80e908e-fe59-4e8c-9769-b9a7c89166fe`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:05:32,231::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:05:32,231::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ab56fb54-a63c-4aa2-842e-810e067d360f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:05:32,232::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:05:32,232::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:05:32,232::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ab56fb54-a63c-4aa2-842e-810e067d360f`::Granted request >Thread-338::DEBUG::2014-05-13 11:05:32,232::task::827::TaskManager.Task::(resourceAcquired) Task=`f80e908e-fe59-4e8c-9769-b9a7c89166fe`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:05:32,233::task::990::TaskManager.Task::(_decref) Task=`f80e908e-fe59-4e8c-9769-b9a7c89166fe`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:05:32,233::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:05:32,234::task::1185::TaskManager.Task::(prepare) Task=`f80e908e-fe59-4e8c-9769-b9a7c89166fe`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:05:32,234::task::595::TaskManager.Task::(_updateState) Task=`f80e908e-fe59-4e8c-9769-b9a7c89166fe`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:05:32,234::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:05:32,234::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:05:32,234::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:05:32,235::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:05:32,235::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:05:32,235::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:05:32,235::task::990::TaskManager.Task::(_decref) Task=`f80e908e-fe59-4e8c-9769-b9a7c89166fe`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:05:34,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:34,150::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310409 s, 33.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:05:34,215::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:05:36,163::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:36,243::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363262 s, 28.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:05:37,235::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:05:38,259::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:38,347::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0455106 s, 22.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:05:40,254::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:05:40,360::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:40,434::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310369 s, 33.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:05:41,792::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:05:41,800::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000605258 s, 6.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:05:42,254::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:05:42,255::task::595::TaskManager.Task::(_updateState) Task=`eca93b5a-061f-4b34-bb01-c050fbc1a268`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:05:42,255::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:05:42,255::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:05:42,256::task::1185::TaskManager.Task::(prepare) Task=`eca93b5a-061f-4b34-bb01-c050fbc1a268`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:05:42,256::task::595::TaskManager.Task::(_updateState) Task=`eca93b5a-061f-4b34-bb01-c050fbc1a268`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:05:42,256::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:05:42,256::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:05:42,256::task::990::TaskManager.Task::(_decref) Task=`eca93b5a-061f-4b34-bb01-c050fbc1a268`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:05:42,261::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:05:42,261::task::595::TaskManager.Task::(_updateState) Task=`6b476008-5a88-47b1-bb24-96f85e867ecd`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:05:42,262::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:05:42,262::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e3329832-b61f-490d-83af-f279ee528258`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:05:42,262::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:05:42,263::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:05:42,263::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e3329832-b61f-490d-83af-f279ee528258`::Granted request >Thread-338::DEBUG::2014-05-13 11:05:42,263::task::827::TaskManager.Task::(resourceAcquired) Task=`6b476008-5a88-47b1-bb24-96f85e867ecd`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:05:42,263::task::990::TaskManager.Task::(_decref) Task=`6b476008-5a88-47b1-bb24-96f85e867ecd`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:05:42,264::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:05:42,264::task::1185::TaskManager.Task::(prepare) Task=`6b476008-5a88-47b1-bb24-96f85e867ecd`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:05:42,264::task::595::TaskManager.Task::(_updateState) Task=`6b476008-5a88-47b1-bb24-96f85e867ecd`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:05:42,265::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:05:42,265::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:05:42,265::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:05:42,265::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:05:42,265::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:05:42,265::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:05:42,266::task::990::TaskManager.Task::(_decref) Task=`6b476008-5a88-47b1-bb24-96f85e867ecd`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:05:42,447::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:42,528::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0371805 s, 27.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:05:43,274::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:05:44,541::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:44,617::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0341647 s, 30.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:05:45,330::task::595::TaskManager.Task::(_updateState) Task=`c1d9f031-68a2-4039-8864-9abee4b5c607`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:05:45,331::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:05:45,331::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000605258', 'lastCheck': '3.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:05:45,331::task::1185::TaskManager.Task::(prepare) Task=`c1d9f031-68a2-4039-8864-9abee4b5c607`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000605258', 'lastCheck': '3.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:05:45,331::task::595::TaskManager.Task::(_updateState) Task=`c1d9f031-68a2-4039-8864-9abee4b5c607`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:05:45,331::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:05:45,332::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:05:45,332::task::990::TaskManager.Task::(_decref) Task=`c1d9f031-68a2-4039-8864-9abee4b5c607`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:05:46,292::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:05:46,630::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:46,708::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035359 s, 29.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:05:48,721::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:48,795::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311531 s, 32.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:05:49,309::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:05:50,808::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:50,898::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0474047 s, 21.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:05:51,807::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:05:51,815::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00056307 s, 7.3 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:05:52,287::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [dc1f215] >Thread-66::DEBUG::2014-05-13 11:05:52,288::task::595::TaskManager.Task::(_updateState) Task=`105feec9-c24d-4f0a-817d-ddaf52c37640`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:05:52,288::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:05:52,288::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:05:52,289::task::1185::TaskManager.Task::(prepare) Task=`105feec9-c24d-4f0a-817d-ddaf52c37640`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:05:52,289::task::595::TaskManager.Task::(_updateState) Task=`105feec9-c24d-4f0a-817d-ddaf52c37640`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:05:52,289::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:05:52,289::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:05:52,289::task::990::TaskManager.Task::(_decref) Task=`105feec9-c24d-4f0a-817d-ddaf52c37640`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:05:52,294::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [dc1f215] >Thread-338::DEBUG::2014-05-13 11:05:52,295::task::595::TaskManager.Task::(_updateState) Task=`92edbc02-6a73-41c2-ae77-7a32a0483b56`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:05:52,295::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:05:52,295::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e20cd663-b2e1-438c-804f-9391645cd628`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:05:52,296::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:05:52,296::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:05:52,296::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e20cd663-b2e1-438c-804f-9391645cd628`::Granted request >Thread-338::DEBUG::2014-05-13 11:05:52,296::task::827::TaskManager.Task::(resourceAcquired) Task=`92edbc02-6a73-41c2-ae77-7a32a0483b56`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:05:52,297::task::990::TaskManager.Task::(_decref) Task=`92edbc02-6a73-41c2-ae77-7a32a0483b56`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:05:52,297::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:05:52,297::task::1185::TaskManager.Task::(prepare) Task=`92edbc02-6a73-41c2-ae77-7a32a0483b56`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:05:52,298::task::595::TaskManager.Task::(_updateState) Task=`92edbc02-6a73-41c2-ae77-7a32a0483b56`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:05:52,298::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:05:52,298::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:05:52,298::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:05:52,298::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:05:52,299::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:05:52,299::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:05:52,299::task::990::TaskManager.Task::(_decref) Task=`92edbc02-6a73-41c2-ae77-7a32a0483b56`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:05:52,327::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:05:52,911::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:52,990::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358673 s, 28.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:05:55,003::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:55,086::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0412608 s, 24.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:05:55,346::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:05:57,099::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:57,179::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359508 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:05:58,367::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:05:59,192::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:05:59,265::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0300839 s, 34.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:06:00,467::task::595::TaskManager.Task::(_updateState) Task=`7ae8950e-c6ba-499b-b25c-7fc08750829c`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:06:00,467::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:06:00,467::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00056307', 'lastCheck': '8.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:06:00,467::task::1185::TaskManager.Task::(prepare) Task=`7ae8950e-c6ba-499b-b25c-7fc08750829c`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00056307', 'lastCheck': '8.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:06:00,468::task::595::TaskManager.Task::(_updateState) Task=`7ae8950e-c6ba-499b-b25c-7fc08750829c`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:06:00,468::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:06:00,468::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:06:00,468::task::990::TaskManager.Task::(_decref) Task=`7ae8950e-c6ba-499b-b25c-7fc08750829c`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:06:01,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:01,352::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307931 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:06:01,386::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:06:01,823::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:06:01,831::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000886616 s, 4.6 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:06:02,318::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:06:02,318::task::595::TaskManager.Task::(_updateState) Task=`c4ae4caf-6d3b-4e7c-b55a-e970e0a41232`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:06:02,318::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:06:02,319::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:06:02,319::task::1185::TaskManager.Task::(prepare) Task=`c4ae4caf-6d3b-4e7c-b55a-e970e0a41232`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:06:02,319::task::595::TaskManager.Task::(_updateState) Task=`c4ae4caf-6d3b-4e7c-b55a-e970e0a41232`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:06:02,319::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:06:02,319::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:06:02,319::task::990::TaskManager.Task::(_decref) Task=`c4ae4caf-6d3b-4e7c-b55a-e970e0a41232`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:06:02,324::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:06:02,324::task::595::TaskManager.Task::(_updateState) Task=`0d5672eb-3a06-46e7-87fa-cd26c13e1c00`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:06:02,325::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:06:02,325::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`537eb174-fa2e-477d-b5e9-53b02d3dd316`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:06:02,325::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:06:02,325::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:06:02,326::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`537eb174-fa2e-477d-b5e9-53b02d3dd316`::Granted request >Thread-338::DEBUG::2014-05-13 11:06:02,326::task::827::TaskManager.Task::(resourceAcquired) Task=`0d5672eb-3a06-46e7-87fa-cd26c13e1c00`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:06:02,326::task::990::TaskManager.Task::(_decref) Task=`0d5672eb-3a06-46e7-87fa-cd26c13e1c00`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:06:02,327::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:06:02,327::task::1185::TaskManager.Task::(prepare) Task=`0d5672eb-3a06-46e7-87fa-cd26c13e1c00`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:06:02,327::task::595::TaskManager.Task::(_updateState) Task=`0d5672eb-3a06-46e7-87fa-cd26c13e1c00`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:06:02,328::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:06:02,328::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:06:02,328::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:06:02,328::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:06:02,328::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:06:02,329::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:06:02,329::task::990::TaskManager.Task::(_decref) Task=`0d5672eb-3a06-46e7-87fa-cd26c13e1c00`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:06:03,365::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:03,775::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.366008 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:06:04,404::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:06:05,788::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:05,868::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359209 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:06:07,422::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:06:07,881::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:07,954::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0301463 s, 34.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:06:09,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:10,038::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0302152 s, 33.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:06:10,441::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:06:11,838::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:06:11,847::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000759784 s, 5.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:06:12,051::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:12,129::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0343823 s, 29.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:06:12,347::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:06:12,347::task::595::TaskManager.Task::(_updateState) Task=`1957be38-46d5-4817-98d9-b3e1815f271f`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:06:12,348::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:06:12,348::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:06:12,348::task::1185::TaskManager.Task::(prepare) Task=`1957be38-46d5-4817-98d9-b3e1815f271f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:06:12,348::task::595::TaskManager.Task::(_updateState) Task=`1957be38-46d5-4817-98d9-b3e1815f271f`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:06:12,348::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:06:12,349::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:06:12,349::task::990::TaskManager.Task::(_decref) Task=`1957be38-46d5-4817-98d9-b3e1815f271f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:06:12,353::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:06:12,354::task::595::TaskManager.Task::(_updateState) Task=`61df601f-60ec-47a7-915d-15cfe2ad931f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:06:12,354::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:06:12,354::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8deeafc8-4daa-4a1c-b24c-61eec0f44c01`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:06:12,354::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:06:12,355::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:06:12,355::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8deeafc8-4daa-4a1c-b24c-61eec0f44c01`::Granted request >Thread-338::DEBUG::2014-05-13 11:06:12,355::task::827::TaskManager.Task::(resourceAcquired) Task=`61df601f-60ec-47a7-915d-15cfe2ad931f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:06:12,355::task::990::TaskManager.Task::(_decref) Task=`61df601f-60ec-47a7-915d-15cfe2ad931f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:06:12,356::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:06:12,356::task::1185::TaskManager.Task::(prepare) Task=`61df601f-60ec-47a7-915d-15cfe2ad931f`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:06:12,357::task::595::TaskManager.Task::(_updateState) Task=`61df601f-60ec-47a7-915d-15cfe2ad931f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:06:12,357::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:06:12,357::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:06:12,357::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:06:12,357::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:06:12,358::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:06:12,358::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:06:12,358::task::990::TaskManager.Task::(_decref) Task=`61df601f-60ec-47a7-915d-15cfe2ad931f`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:06:13,461::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:06:14,142::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:14,213::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0297184 s, 34.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:06:15,607::task::595::TaskManager.Task::(_updateState) Task=`8f0e975d-d6f9-4604-bd59-99c31fcd9491`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:06:15,608::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:06:15,608::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000759784', 'lastCheck': '3.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:06:15,608::task::1185::TaskManager.Task::(prepare) Task=`8f0e975d-d6f9-4604-bd59-99c31fcd9491`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000759784', 'lastCheck': '3.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:06:15,608::task::595::TaskManager.Task::(_updateState) Task=`8f0e975d-d6f9-4604-bd59-99c31fcd9491`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:06:15,608::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:06:15,609::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:06:15,609::task::990::TaskManager.Task::(_decref) Task=`8f0e975d-d6f9-4604-bd59-99c31fcd9491`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:06:16,226::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:16,311::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.041704 s, 24.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:06:16,480::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:06:18,324::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:18,407::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.03869 s, 26.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:06:19,499::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:06:20,420::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:20,493::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312454 s, 32.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:06:21,854::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:06:21,863::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00071474 s, 5.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:06:22,376::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:06:22,377::task::595::TaskManager.Task::(_updateState) Task=`7bc4edb1-bc0d-41a0-9480-47850566c958`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:06:22,377::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:06:22,377::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:06:22,377::task::1185::TaskManager.Task::(prepare) Task=`7bc4edb1-bc0d-41a0-9480-47850566c958`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:06:22,377::task::595::TaskManager.Task::(_updateState) Task=`7bc4edb1-bc0d-41a0-9480-47850566c958`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:06:22,378::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:06:22,378::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:06:22,378::task::990::TaskManager.Task::(_decref) Task=`7bc4edb1-bc0d-41a0-9480-47850566c958`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:06:22,382::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:06:22,383::task::595::TaskManager.Task::(_updateState) Task=`092320b8-6adb-410a-aac1-a29fd3b99bcd`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:06:22,383::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:06:22,383::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6cca94c3-310f-444e-8358-ab5083415342`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:06:22,384::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:06:22,384::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:06:22,384::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6cca94c3-310f-444e-8358-ab5083415342`::Granted request >Thread-338::DEBUG::2014-05-13 11:06:22,384::task::827::TaskManager.Task::(resourceAcquired) Task=`092320b8-6adb-410a-aac1-a29fd3b99bcd`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:06:22,385::task::990::TaskManager.Task::(_decref) Task=`092320b8-6adb-410a-aac1-a29fd3b99bcd`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:06:22,385::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:06:22,385::task::1185::TaskManager.Task::(prepare) Task=`092320b8-6adb-410a-aac1-a29fd3b99bcd`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:06:22,386::task::595::TaskManager.Task::(_updateState) Task=`092320b8-6adb-410a-aac1-a29fd3b99bcd`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:06:22,386::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:06:22,386::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:06:22,386::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:06:22,386::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:06:22,387::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:06:22,387::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:06:22,387::task::990::TaskManager.Task::(_decref) Task=`092320b8-6adb-410a-aac1-a29fd3b99bcd`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:06:22,507::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:06:22,520::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:06:22,583::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352424 s, 29.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:06:24,596::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:24,677::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360435 s, 28.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:06:25,540::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:06:26,690::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:26,772::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303511 s, 33.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:06:28,559::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:06:28,786::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:28,859::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305806 s, 33.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:06:30,743::task::595::TaskManager.Task::(_updateState) Task=`e7cfc831-b0ef-406b-81b0-0d9b258f6b31`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:06:30,743::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:06:30,743::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00071474', 'lastCheck': '8.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:06:30,744::task::1185::TaskManager.Task::(prepare) Task=`e7cfc831-b0ef-406b-81b0-0d9b258f6b31`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00071474', 'lastCheck': '8.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:06:30,744::task::595::TaskManager.Task::(_updateState) Task=`e7cfc831-b0ef-406b-81b0-0d9b258f6b31`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:06:30,744::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:06:30,744::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:06:30,744::task::990::TaskManager.Task::(_decref) Task=`e7cfc831-b0ef-406b-81b0-0d9b258f6b31`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:06:30,872::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:30,947::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0330027 s, 31.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:06:31,575::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:06:31,870::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:06:31,878::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000690281 s, 5.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:06:32,405::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:06:32,406::task::595::TaskManager.Task::(_updateState) Task=`4808aff0-c7bf-40c9-93c6-b655f96dfb9e`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:06:32,406::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:06:32,406::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:06:32,406::task::1185::TaskManager.Task::(prepare) Task=`4808aff0-c7bf-40c9-93c6-b655f96dfb9e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:06:32,407::task::595::TaskManager.Task::(_updateState) Task=`4808aff0-c7bf-40c9-93c6-b655f96dfb9e`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:06:32,407::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:06:32,407::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:06:32,407::task::990::TaskManager.Task::(_decref) Task=`4808aff0-c7bf-40c9-93c6-b655f96dfb9e`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:06:32,412::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:06:32,412::task::595::TaskManager.Task::(_updateState) Task=`9c1e4ab8-53cc-47fe-8fc5-eb82b80301e7`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:06:32,412::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:06:32,413::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`606edae3-06fd-4ba4-ba70-09a3068f8840`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:06:32,413::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:06:32,413::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:06:32,413::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`606edae3-06fd-4ba4-ba70-09a3068f8840`::Granted request >Thread-338::DEBUG::2014-05-13 11:06:32,414::task::827::TaskManager.Task::(resourceAcquired) Task=`9c1e4ab8-53cc-47fe-8fc5-eb82b80301e7`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:06:32,414::task::990::TaskManager.Task::(_decref) Task=`9c1e4ab8-53cc-47fe-8fc5-eb82b80301e7`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:06:32,415::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:06:32,415::task::1185::TaskManager.Task::(prepare) Task=`9c1e4ab8-53cc-47fe-8fc5-eb82b80301e7`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:06:32,415::task::595::TaskManager.Task::(_updateState) Task=`9c1e4ab8-53cc-47fe-8fc5-eb82b80301e7`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:06:32,415::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:06:32,415::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:06:32,416::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:06:32,416::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:06:32,416::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:06:32,416::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:06:32,416::task::990::TaskManager.Task::(_decref) Task=`9c1e4ab8-53cc-47fe-8fc5-eb82b80301e7`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:06:32,960::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:33,057::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.052902 s, 19.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:06:34,596::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:06:35,071::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:35,166::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311645 s, 32.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:06:37,179::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:37,258::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361902 s, 28.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:06:37,616::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:06:39,271::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:39,353::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.038165 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:06:40,635::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:06:41,365::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:41,439::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303456 s, 33.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:06:41,886::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:06:41,894::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000632222 s, 6.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:06:42,436::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:06:42,436::task::595::TaskManager.Task::(_updateState) Task=`b6a74d4f-756e-47af-a6ed-e27ffe10f751`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:06:42,437::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:06:42,437::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:06:42,437::task::1185::TaskManager.Task::(prepare) Task=`b6a74d4f-756e-47af-a6ed-e27ffe10f751`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:06:42,437::task::595::TaskManager.Task::(_updateState) Task=`b6a74d4f-756e-47af-a6ed-e27ffe10f751`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:06:42,438::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:06:42,438::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:06:42,438::task::990::TaskManager.Task::(_decref) Task=`b6a74d4f-756e-47af-a6ed-e27ffe10f751`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:06:42,443::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:06:42,443::task::595::TaskManager.Task::(_updateState) Task=`87290d44-74aa-40db-9274-2ad0a19d0a0d`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:06:42,443::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:06:42,444::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5c410f15-2995-4473-9b5f-522330eb5032`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:06:42,444::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:06:42,444::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:06:42,444::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5c410f15-2995-4473-9b5f-522330eb5032`::Granted request >Thread-338::DEBUG::2014-05-13 11:06:42,445::task::827::TaskManager.Task::(resourceAcquired) Task=`87290d44-74aa-40db-9274-2ad0a19d0a0d`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:06:42,445::task::990::TaskManager.Task::(_decref) Task=`87290d44-74aa-40db-9274-2ad0a19d0a0d`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:06:42,446::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:06:42,446::task::1185::TaskManager.Task::(prepare) Task=`87290d44-74aa-40db-9274-2ad0a19d0a0d`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:06:42,446::task::595::TaskManager.Task::(_updateState) Task=`87290d44-74aa-40db-9274-2ad0a19d0a0d`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:06:42,446::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:06:42,446::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:06:42,447::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:06:42,447::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:06:42,447::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:06:42,447::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:06:42,447::task::990::TaskManager.Task::(_decref) Task=`87290d44-74aa-40db-9274-2ad0a19d0a0d`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:06:43,452::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:43,526::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306127 s, 33.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:06:43,658::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:06:45,539::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:45,628::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0459594 s, 22.3 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:06:45,877::task::595::TaskManager.Task::(_updateState) Task=`65d94c42-9701-4b75-ba97-409d8164d5e8`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:06:45,877::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:06:45,878::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000632222', 'lastCheck': '3.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:06:45,878::task::1185::TaskManager.Task::(prepare) Task=`65d94c42-9701-4b75-ba97-409d8164d5e8`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000632222', 'lastCheck': '3.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:06:45,878::task::595::TaskManager.Task::(_updateState) Task=`65d94c42-9701-4b75-ba97-409d8164d5e8`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:06:45,878::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:06:45,878::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:06:45,878::task::990::TaskManager.Task::(_decref) Task=`65d94c42-9701-4b75-ba97-409d8164d5e8`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:06:46,680::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:06:47,641::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:47,721::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355365 s, 28.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:06:49,698::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:06:49,734::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:49,813::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0348365 s, 29.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:06:51,826::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:51,907::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0369998 s, 27.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:06:51,939::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:06:51,947::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000521952 s, 7.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:06:52,465::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1ea398d3] >Thread-66::DEBUG::2014-05-13 11:06:52,466::task::595::TaskManager.Task::(_updateState) Task=`5c526381-bd1e-4195-b11d-f40e01c803b4`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:06:52,466::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:06:52,466::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:06:52,466::task::1185::TaskManager.Task::(prepare) Task=`5c526381-bd1e-4195-b11d-f40e01c803b4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:06:52,467::task::595::TaskManager.Task::(_updateState) Task=`5c526381-bd1e-4195-b11d-f40e01c803b4`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:06:52,467::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:06:52,467::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:06:52,467::task::990::TaskManager.Task::(_decref) Task=`5c526381-bd1e-4195-b11d-f40e01c803b4`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:06:52,472::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1ea398d3] >Thread-338::DEBUG::2014-05-13 11:06:52,473::task::595::TaskManager.Task::(_updateState) Task=`8c9ed6ce-c899-441e-bebd-63e691111750`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:06:52,473::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:06:52,473::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d3522372-83d5-4292-96fc-483f6129d787`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:06:52,474::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:06:52,474::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:06:52,474::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d3522372-83d5-4292-96fc-483f6129d787`::Granted request >Thread-338::DEBUG::2014-05-13 11:06:52,474::task::827::TaskManager.Task::(resourceAcquired) Task=`8c9ed6ce-c899-441e-bebd-63e691111750`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:06:52,475::task::990::TaskManager.Task::(_decref) Task=`8c9ed6ce-c899-441e-bebd-63e691111750`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:06:52,475::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:06:52,476::task::1185::TaskManager.Task::(prepare) Task=`8c9ed6ce-c899-441e-bebd-63e691111750`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:06:52,476::task::595::TaskManager.Task::(_updateState) Task=`8c9ed6ce-c899-441e-bebd-63e691111750`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:06:52,476::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:06:52,476::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:06:52,476::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:06:52,477::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:06:52,477::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:06:52,477::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:06:52,477::task::990::TaskManager.Task::(_decref) Task=`8c9ed6ce-c899-441e-bebd-63e691111750`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:06:52,716::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:06:53,920::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:54,000::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362477 s, 28.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:06:55,734::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:06:56,012::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:56,087::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315016 s, 32.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:06:58,100::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:06:58,175::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316969 s, 32.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:06:58,751::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:07:00,188::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:00,549::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.318491 s, 3.2 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:07:01,018::task::595::TaskManager.Task::(_updateState) Task=`91faac21-e504-4718-9098-98c439869616`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:07:01,018::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:07:01,018::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000521952', 'lastCheck': '9.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:07:01,019::task::1185::TaskManager.Task::(prepare) Task=`91faac21-e504-4718-9098-98c439869616`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000521952', 'lastCheck': '9.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:07:01,019::task::595::TaskManager.Task::(_updateState) Task=`91faac21-e504-4718-9098-98c439869616`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:07:01,019::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:07:01,019::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:07:01,019::task::990::TaskManager.Task::(_decref) Task=`91faac21-e504-4718-9098-98c439869616`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:07:01,769::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:07:01,954::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:07:01,963::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000733301 s, 5.6 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:07:02,497::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:07:02,497::task::595::TaskManager.Task::(_updateState) Task=`a7d867bf-79f1-4ed9-bd1e-72569d7d96f3`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:07:02,498::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:07:02,498::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:07:02,498::task::1185::TaskManager.Task::(prepare) Task=`a7d867bf-79f1-4ed9-bd1e-72569d7d96f3`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:07:02,498::task::595::TaskManager.Task::(_updateState) Task=`a7d867bf-79f1-4ed9-bd1e-72569d7d96f3`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:07:02,498::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:07:02,499::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:07:02,499::task::990::TaskManager.Task::(_decref) Task=`a7d867bf-79f1-4ed9-bd1e-72569d7d96f3`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:07:02,503::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:07:02,504::task::595::TaskManager.Task::(_updateState) Task=`34b9c492-ee1d-43ef-ade2-7a17ea4de1ca`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:07:02,504::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:07:02,504::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`88bc3511-56b8-4fad-80bc-28a3d90c00ae`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:07:02,504::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:07:02,505::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:07:02,505::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`88bc3511-56b8-4fad-80bc-28a3d90c00ae`::Granted request >Thread-338::DEBUG::2014-05-13 11:07:02,505::task::827::TaskManager.Task::(resourceAcquired) Task=`34b9c492-ee1d-43ef-ade2-7a17ea4de1ca`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:07:02,506::task::990::TaskManager.Task::(_decref) Task=`34b9c492-ee1d-43ef-ade2-7a17ea4de1ca`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:07:02,506::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:07:02,506::task::1185::TaskManager.Task::(prepare) Task=`34b9c492-ee1d-43ef-ade2-7a17ea4de1ca`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:07:02,507::task::595::TaskManager.Task::(_updateState) Task=`34b9c492-ee1d-43ef-ade2-7a17ea4de1ca`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:07:02,507::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:07:02,507::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:07:02,507::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:07:02,507::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:07:02,508::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:07:02,508::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:07:02,508::task::990::TaskManager.Task::(_decref) Task=`34b9c492-ee1d-43ef-ade2-7a17ea4de1ca`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:07:02,562::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:02,642::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036284 s, 28.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:07:04,656::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:04,730::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305682 s, 33.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:07:04,791::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:07:06,743::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:06,820::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0336186 s, 30.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:07:07,809::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:07:08,833::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:08,913::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363049 s, 28.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:07:10,827::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:07:10,926::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:11,000::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306961 s, 33.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:07:11,970::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:07:11,979::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000688548 s, 5.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:07:12,528::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:07:12,529::task::595::TaskManager.Task::(_updateState) Task=`97f3d7cf-1170-4549-a156-c767f98f3e07`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:07:12,529::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:07:12,529::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:07:12,529::task::1185::TaskManager.Task::(prepare) Task=`97f3d7cf-1170-4549-a156-c767f98f3e07`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:07:12,529::task::595::TaskManager.Task::(_updateState) Task=`97f3d7cf-1170-4549-a156-c767f98f3e07`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:07:12,530::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:07:12,530::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:07:12,530::task::990::TaskManager.Task::(_decref) Task=`97f3d7cf-1170-4549-a156-c767f98f3e07`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:07:12,534::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:07:12,535::task::595::TaskManager.Task::(_updateState) Task=`9ea85e3f-4baf-4597-b079-f6a1a9544a98`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:07:12,535::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:07:12,536::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`850711a6-acb5-4e88-b018-747e389bc64e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:07:12,536::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:07:12,537::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:07:12,537::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`850711a6-acb5-4e88-b018-747e389bc64e`::Granted request >Thread-338::DEBUG::2014-05-13 11:07:12,537::task::827::TaskManager.Task::(resourceAcquired) Task=`9ea85e3f-4baf-4597-b079-f6a1a9544a98`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:07:12,537::task::990::TaskManager.Task::(_decref) Task=`9ea85e3f-4baf-4597-b079-f6a1a9544a98`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:07:12,538::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:07:12,538::task::1185::TaskManager.Task::(prepare) Task=`9ea85e3f-4baf-4597-b079-f6a1a9544a98`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:07:12,539::task::595::TaskManager.Task::(_updateState) Task=`9ea85e3f-4baf-4597-b079-f6a1a9544a98`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:07:12,539::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:07:12,539::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:07:12,539::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:07:12,539::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:07:12,540::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:07:12,540::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:07:12,540::task::990::TaskManager.Task::(_decref) Task=`9ea85e3f-4baf-4597-b079-f6a1a9544a98`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:07:13,013::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:13,103::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0449114 s, 22.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:07:13,850::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:07:15,116::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:15,198::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0366421 s, 27.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:07:16,158::task::595::TaskManager.Task::(_updateState) Task=`89ac2f78-88ba-47f8-96f8-a689b755da74`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:07:16,158::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:07:16,158::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000688548', 'lastCheck': '4.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:07:16,159::task::1185::TaskManager.Task::(prepare) Task=`89ac2f78-88ba-47f8-96f8-a689b755da74`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000688548', 'lastCheck': '4.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:07:16,159::task::595::TaskManager.Task::(_updateState) Task=`89ac2f78-88ba-47f8-96f8-a689b755da74`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:07:16,159::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:07:16,159::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:07:16,159::task::990::TaskManager.Task::(_decref) Task=`89ac2f78-88ba-47f8-96f8-a689b755da74`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:07:16,868::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:07:17,211::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:17,308::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0540617 s, 18.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:07:19,321::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:19,396::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0329715 s, 31.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:07:19,886::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:07:21,410::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:21,496::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0421509 s, 24.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:07:21,986::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:07:21,996::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000640341 s, 6.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:07:22,564::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:07:22,565::task::595::TaskManager.Task::(_updateState) Task=`6a520890-db99-47c6-8294-c90463dc6c21`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:07:22,565::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:07:22,565::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:07:22,565::task::1185::TaskManager.Task::(prepare) Task=`6a520890-db99-47c6-8294-c90463dc6c21`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:07:22,566::task::595::TaskManager.Task::(_updateState) Task=`6a520890-db99-47c6-8294-c90463dc6c21`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:07:22,566::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:07:22,566::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:07:22,566::task::990::TaskManager.Task::(_decref) Task=`6a520890-db99-47c6-8294-c90463dc6c21`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:07:22,571::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:07:22,571::task::595::TaskManager.Task::(_updateState) Task=`42214f33-b62e-4ac8-8818-6b3c37574d22`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:07:22,571::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:07:22,572::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`19f6dd8e-1425-4846-8e85-1cdd5d56ab96`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:07:22,572::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:07:22,572::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:07:22,572::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`19f6dd8e-1425-4846-8e85-1cdd5d56ab96`::Granted request >Thread-338::DEBUG::2014-05-13 11:07:22,573::task::827::TaskManager.Task::(resourceAcquired) Task=`42214f33-b62e-4ac8-8818-6b3c37574d22`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:07:22,573::task::990::TaskManager.Task::(_decref) Task=`42214f33-b62e-4ac8-8818-6b3c37574d22`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:07:22,574::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:07:22,574::task::1185::TaskManager.Task::(prepare) Task=`42214f33-b62e-4ac8-8818-6b3c37574d22`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:07:22,574::task::595::TaskManager.Task::(_updateState) Task=`42214f33-b62e-4ac8-8818-6b3c37574d22`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:07:22,574::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:07:22,574::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:07:22,575::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:07:22,575::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:07:22,575::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:07:22,575::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:07:22,575::task::990::TaskManager.Task::(_decref) Task=`42214f33-b62e-4ac8-8818-6b3c37574d22`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:07:22,905::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:07:23,509::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:23,588::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356868 s, 28.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:07:25,601::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:25,689::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0444372 s, 23.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:07:25,925::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:07:27,702::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:27,782::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362504 s, 28.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:07:28,943::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:07:29,793::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:29,869::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314136 s, 32.6 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:07:31,290::task::595::TaskManager.Task::(_updateState) Task=`ba50bde2-f77a-49be-9e20-58b637a06d2a`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:07:31,290::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:07:31,290::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000640341', 'lastCheck': '9.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:07:31,291::task::1185::TaskManager.Task::(prepare) Task=`ba50bde2-f77a-49be-9e20-58b637a06d2a`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000640341', 'lastCheck': '9.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:07:31,291::task::595::TaskManager.Task::(_updateState) Task=`ba50bde2-f77a-49be-9e20-58b637a06d2a`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:07:31,291::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:07:31,291::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:07:31,291::task::990::TaskManager.Task::(_decref) Task=`ba50bde2-f77a-49be-9e20-58b637a06d2a`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:07:31,882::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:31,956::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305696 s, 33.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:07:31,970::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:07:32,003::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:07:32,012::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.0004894 s, 8.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:07:32,599::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [7d725256] >Thread-66::DEBUG::2014-05-13 11:07:32,599::task::595::TaskManager.Task::(_updateState) Task=`6f837289-73d9-4cae-bcd4-3280edbd5c21`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:07:32,599::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:07:32,600::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:07:32,600::task::1185::TaskManager.Task::(prepare) Task=`6f837289-73d9-4cae-bcd4-3280edbd5c21`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:07:32,600::task::595::TaskManager.Task::(_updateState) Task=`6f837289-73d9-4cae-bcd4-3280edbd5c21`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:07:32,600::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:07:32,600::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:07:32,601::task::990::TaskManager.Task::(_decref) Task=`6f837289-73d9-4cae-bcd4-3280edbd5c21`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:07:32,606::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [7d725256] >Thread-338::DEBUG::2014-05-13 11:07:32,606::task::595::TaskManager.Task::(_updateState) Task=`56d72d82-63b6-4fcc-a292-d54f4798a421`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:07:32,607::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:07:32,607::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`01eb2963-db66-4909-ad6b-9a5ee32144b9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:07:32,607::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:07:32,608::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:07:32,608::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`01eb2963-db66-4909-ad6b-9a5ee32144b9`::Granted request >Thread-338::DEBUG::2014-05-13 11:07:32,608::task::827::TaskManager.Task::(resourceAcquired) Task=`56d72d82-63b6-4fcc-a292-d54f4798a421`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:07:32,608::task::990::TaskManager.Task::(_decref) Task=`56d72d82-63b6-4fcc-a292-d54f4798a421`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:07:32,609::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:07:32,609::task::1185::TaskManager.Task::(prepare) Task=`56d72d82-63b6-4fcc-a292-d54f4798a421`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:07:32,609::task::595::TaskManager.Task::(_updateState) Task=`56d72d82-63b6-4fcc-a292-d54f4798a421`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:07:32,610::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:07:32,610::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:07:32,610::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:07:32,610::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:07:32,610::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:07:32,610::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:07:32,611::task::990::TaskManager.Task::(_decref) Task=`56d72d82-63b6-4fcc-a292-d54f4798a421`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:07:33,970::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:34,045::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309468 s, 33.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:07:34,988::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:07:36,057::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:36,137::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353295 s, 29.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:07:38,007::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:07:38,150::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:38,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352537 s, 29.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:07:40,242::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:40,316::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0304252 s, 33.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:07:41,025::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:07:42,019::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:07:42,027::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000709201 s, 5.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:07:42,329::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:42,421::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0480751 s, 21.3 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:07:42,630::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:07:42,631::task::595::TaskManager.Task::(_updateState) Task=`7f417760-0fd5-463c-9176-11cc2371eac5`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:07:42,631::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:07:42,631::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:07:42,631::task::1185::TaskManager.Task::(prepare) Task=`7f417760-0fd5-463c-9176-11cc2371eac5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:07:42,631::task::595::TaskManager.Task::(_updateState) Task=`7f417760-0fd5-463c-9176-11cc2371eac5`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:07:42,632::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:07:42,632::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:07:42,632::task::990::TaskManager.Task::(_decref) Task=`7f417760-0fd5-463c-9176-11cc2371eac5`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:07:42,638::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:07:42,638::task::595::TaskManager.Task::(_updateState) Task=`b37a77df-879a-4b79-9e4c-fe37f1711a6c`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:07:42,638::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:07:42,639::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1550346e-c138-4bec-a31a-23ede64ba5fc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:07:42,639::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:07:42,639::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:07:42,639::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1550346e-c138-4bec-a31a-23ede64ba5fc`::Granted request >Thread-338::DEBUG::2014-05-13 11:07:42,640::task::827::TaskManager.Task::(resourceAcquired) Task=`b37a77df-879a-4b79-9e4c-fe37f1711a6c`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:07:42,640::task::990::TaskManager.Task::(_decref) Task=`b37a77df-879a-4b79-9e4c-fe37f1711a6c`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:07:42,641::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:07:42,641::task::1185::TaskManager.Task::(prepare) Task=`b37a77df-879a-4b79-9e4c-fe37f1711a6c`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:07:42,641::task::595::TaskManager.Task::(_updateState) Task=`b37a77df-879a-4b79-9e4c-fe37f1711a6c`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:07:42,641::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:07:42,641::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:07:42,642::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:07:42,642::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:07:42,642::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:07:42,642::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:07:42,642::task::990::TaskManager.Task::(_decref) Task=`b37a77df-879a-4b79-9e4c-fe37f1711a6c`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:07:44,044::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:07:44,434::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:44,508::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0304192 s, 33.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:07:46,521::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:46,597::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315463 s, 32.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:07:46,695::task::595::TaskManager.Task::(_updateState) Task=`1dd1b303-f08c-4ed5-b107-8321aeac6483`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:07:46,696::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:07:46,696::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000709201', 'lastCheck': '4.7', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:07:46,696::task::1185::TaskManager.Task::(prepare) Task=`1dd1b303-f08c-4ed5-b107-8321aeac6483`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000709201', 'lastCheck': '4.7', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:07:46,696::task::595::TaskManager.Task::(_updateState) Task=`1dd1b303-f08c-4ed5-b107-8321aeac6483`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:07:46,697::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:07:46,697::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:07:46,697::task::990::TaskManager.Task::(_decref) Task=`1dd1b303-f08c-4ed5-b107-8321aeac6483`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:07:47,064::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:07:48,610::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:48,690::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035331 s, 29.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:07:50,083::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:07:50,703::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:50,779::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0328283 s, 31.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:07:52,034::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:07:52,042::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000701583 s, 5.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:07:52,663::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5fad7764] >Thread-66::DEBUG::2014-05-13 11:07:52,663::task::595::TaskManager.Task::(_updateState) Task=`99323db6-b4f7-49a5-a08c-14529a1d0ef4`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:07:52,664::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:07:52,664::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:07:52,664::task::1185::TaskManager.Task::(prepare) Task=`99323db6-b4f7-49a5-a08c-14529a1d0ef4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:07:52,664::task::595::TaskManager.Task::(_updateState) Task=`99323db6-b4f7-49a5-a08c-14529a1d0ef4`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:07:52,665::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:07:52,665::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:07:52,665::task::990::TaskManager.Task::(_decref) Task=`99323db6-b4f7-49a5-a08c-14529a1d0ef4`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:07:52,669::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5fad7764] >Thread-338::DEBUG::2014-05-13 11:07:52,670::task::595::TaskManager.Task::(_updateState) Task=`7024fa58-db36-4b95-8bea-b68b4efdd9c6`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:07:52,670::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:07:52,670::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`536cbb5d-54ba-4da6-8788-93be4e821766`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:07:52,671::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:07:52,671::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:07:52,671::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`536cbb5d-54ba-4da6-8788-93be4e821766`::Granted request >Thread-338::DEBUG::2014-05-13 11:07:52,672::task::827::TaskManager.Task::(resourceAcquired) Task=`7024fa58-db36-4b95-8bea-b68b4efdd9c6`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:07:52,672::task::990::TaskManager.Task::(_decref) Task=`7024fa58-db36-4b95-8bea-b68b4efdd9c6`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:07:52,672::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:07:52,673::task::1185::TaskManager.Task::(prepare) Task=`7024fa58-db36-4b95-8bea-b68b4efdd9c6`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:07:52,673::task::595::TaskManager.Task::(_updateState) Task=`7024fa58-db36-4b95-8bea-b68b4efdd9c6`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:07:52,673::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:07:52,673::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:07:52,673::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:07:52,674::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:07:52,674::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:07:52,674::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:07:52,674::task::990::TaskManager.Task::(_decref) Task=`7024fa58-db36-4b95-8bea-b68b4efdd9c6`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:07:52,792::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:52,879::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0421977 s, 24.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:07:53,104::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:07:54,891::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:54,977::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0422306 s, 24.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:07:56,123::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:07:56,989::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:07:57,071::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0378968 s, 27.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:07:59,084::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:07:59,140::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:07:59,162::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309825 s, 33.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:08:01,175::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:08:01,249::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323008 s, 31.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:08:01,866::task::595::TaskManager.Task::(_updateState) Task=`47369996-fe96-45e5-8f2f-e39fde194856`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:08:01,867::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:08:01,867::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000701583', 'lastCheck': '9.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:08:01,867::task::1185::TaskManager.Task::(prepare) Task=`47369996-fe96-45e5-8f2f-e39fde194856`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000701583', 'lastCheck': '9.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:08:01,867::task::595::TaskManager.Task::(_updateState) Task=`47369996-fe96-45e5-8f2f-e39fde194856`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:08:01,867::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:08:01,868::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:08:01,868::task::990::TaskManager.Task::(_decref) Task=`47369996-fe96-45e5-8f2f-e39fde194856`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 11:08:02,050::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:08:02,058::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000588079 s, 7.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:08:02,157::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:08:02,695::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:08:02,695::task::595::TaskManager.Task::(_updateState) Task=`2024a748-8aa1-49b1-ae9a-e6e56190fa1c`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:08:02,696::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:08:02,696::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:08:02,696::task::1185::TaskManager.Task::(prepare) Task=`2024a748-8aa1-49b1-ae9a-e6e56190fa1c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:08:02,696::task::595::TaskManager.Task::(_updateState) Task=`2024a748-8aa1-49b1-ae9a-e6e56190fa1c`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:08:02,697::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:08:02,697::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:08:02,697::task::990::TaskManager.Task::(_decref) Task=`2024a748-8aa1-49b1-ae9a-e6e56190fa1c`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:08:02,702::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:08:02,702::task::595::TaskManager.Task::(_updateState) Task=`87c1a627-4997-48de-9f72-c682de332dcc`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:08:02,702::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:08:02,703::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c9ace0cd-280c-49a4-8ecc-fcddbbdca8b0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:08:02,703::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:08:02,703::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:08:02,703::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c9ace0cd-280c-49a4-8ecc-fcddbbdca8b0`::Granted request >Thread-338::DEBUG::2014-05-13 11:08:02,704::task::827::TaskManager.Task::(resourceAcquired) Task=`87c1a627-4997-48de-9f72-c682de332dcc`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:08:02,704::task::990::TaskManager.Task::(_decref) Task=`87c1a627-4997-48de-9f72-c682de332dcc`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:08:02,705::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:08:02,705::task::1185::TaskManager.Task::(prepare) Task=`87c1a627-4997-48de-9f72-c682de332dcc`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:08:02,705::task::595::TaskManager.Task::(_updateState) Task=`87c1a627-4997-48de-9f72-c682de332dcc`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:08:02,705::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:08:02,705::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:08:02,706::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:08:02,706::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:08:02,706::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:08:02,706::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:08:02,706::task::990::TaskManager.Task::(_decref) Task=`87c1a627-4997-48de-9f72-c682de332dcc`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:08:03,262::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:08:03,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382401 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:08:05,175::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:08:05,357::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:08:05,442::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0418449 s, 24.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:08:07,455::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:08:07,913::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.412321 s, 2.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:08:08,194::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:08:09,926::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:08:09,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0299631 s, 34.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:08:11,219::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:08:12,012::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 11:08:12,087::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:08:12,106::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000776351 s, 5.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:08:12,157::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379868 s, 27.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:08:12,725::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:08:12,726::task::595::TaskManager.Task::(_updateState) Task=`269232ec-3ebe-490a-84e4-1b012d2ca022`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:08:12,726::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:08:12,726::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:08:12,726::task::1185::TaskManager.Task::(prepare) Task=`269232ec-3ebe-490a-84e4-1b012d2ca022`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:08:12,727::task::595::TaskManager.Task::(_updateState) Task=`269232ec-3ebe-490a-84e4-1b012d2ca022`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:08:12,727::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:08:12,727::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:08:12,727::task::990::TaskManager.Task::(_decref) Task=`269232ec-3ebe-490a-84e4-1b012d2ca022`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:08:12,732::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:08:12,733::task::595::TaskManager.Task::(_updateState) Task=`e73d1fce-b360-4628-b8bc-1cb5819a5bfc`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:08:12,733::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:08:12,733::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fbc2bc2c-6262-4c2f-8f27-453c225b9081`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:08:12,733::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:08:12,734::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:08:12,734::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fbc2bc2c-6262-4c2f-8f27-453c225b9081`::Granted request >Thread-338::DEBUG::2014-05-13 11:08:12,734::task::827::TaskManager.Task::(resourceAcquired) Task=`e73d1fce-b360-4628-b8bc-1cb5819a5bfc`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:08:12,735::task::990::TaskManager.Task::(_decref) Task=`e73d1fce-b360-4628-b8bc-1cb5819a5bfc`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:08:12,735::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:08:12,736::task::1185::TaskManager.Task::(prepare) Task=`e73d1fce-b360-4628-b8bc-1cb5819a5bfc`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:08:12,736::task::595::TaskManager.Task::(_updateState) Task=`e73d1fce-b360-4628-b8bc-1cb5819a5bfc`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:08:12,736::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:08:12,736::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:08:12,736::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:08:12,737::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:08:12,737::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:08:12,737::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:08:12,737::task::990::TaskManager.Task::(_decref) Task=`e73d1fce-b360-4628-b8bc-1cb5819a5bfc`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:08:14,170::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:08:14,238::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:08:14,248::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318678 s, 32.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:08:16,261::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:08:16,335::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317853 s, 32.2 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:08:17,015::task::595::TaskManager.Task::(_updateState) Task=`f88c594c-cae4-433f-9d12-632a9a78d3b8`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:08:17,016::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:08:17,016::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000776351', 'lastCheck': '4.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:08:17,016::task::1185::TaskManager.Task::(prepare) Task=`f88c594c-cae4-433f-9d12-632a9a78d3b8`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000776351', 'lastCheck': '4.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:08:17,016::task::595::TaskManager.Task::(_updateState) Task=`f88c594c-cae4-433f-9d12-632a9a78d3b8`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:08:17,016::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:08:17,017::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:08:17,017::task::990::TaskManager.Task::(_decref) Task=`f88c594c-cae4-433f-9d12-632a9a78d3b8`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:08:17,256::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:08:18,348::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:08:18,429::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359675 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:08:20,275::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:08:20,442::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:08:20,844::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.358609 s, 2.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:08:22,133::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:08:22,143::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00240148 s, 1.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:08:22,756::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:08:22,757::task::595::TaskManager.Task::(_updateState) Task=`c3353b2c-62fd-4a67-9612-4a52d0bd86e4`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:08:22,757::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:08:22,757::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:08:22,757::task::1185::TaskManager.Task::(prepare) Task=`c3353b2c-62fd-4a67-9612-4a52d0bd86e4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:08:22,758::task::595::TaskManager.Task::(_updateState) Task=`c3353b2c-62fd-4a67-9612-4a52d0bd86e4`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:08:22,758::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:08:22,758::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:08:22,758::task::990::TaskManager.Task::(_decref) Task=`c3353b2c-62fd-4a67-9612-4a52d0bd86e4`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:08:22,764::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:08:22,764::task::595::TaskManager.Task::(_updateState) Task=`efd77af3-015a-413b-892b-7d64f951c533`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:08:22,764::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:08:22,765::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`40b6b04d-11f9-4e1c-9fad-f80ebf9cca0c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:08:22,765::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:08:22,765::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:08:22,765::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`40b6b04d-11f9-4e1c-9fad-f80ebf9cca0c`::Granted request >Thread-338::DEBUG::2014-05-13 11:08:22,767::task::827::TaskManager.Task::(resourceAcquired) Task=`efd77af3-015a-413b-892b-7d64f951c533`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:08:22,767::task::990::TaskManager.Task::(_decref) Task=`efd77af3-015a-413b-892b-7d64f951c533`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:08:22,767::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:08:22,768::task::1185::TaskManager.Task::(prepare) Task=`efd77af3-015a-413b-892b-7d64f951c533`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:08:22,768::task::595::TaskManager.Task::(_updateState) Task=`efd77af3-015a-413b-892b-7d64f951c533`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:08:22,768::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:08:22,768::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:08:22,768::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:08:22,769::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:08:22,769::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:08:22,769::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:08:22,769::task::990::TaskManager.Task::(_decref) Task=`efd77af3-015a-413b-892b-7d64f951c533`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:08:22,857::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:08:22,941::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0409106 s, 25.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:08:23,297::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:08:24,954::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:08:25,028::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310758 s, 33.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:08:26,315::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:08:27,041::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:08:27,122::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373325 s, 27.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:08:29,135::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:08:29,211::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315389 s, 32.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:08:29,333::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:08:31,224::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:08:31,623::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.357382 s, 2.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:08:32,150::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-66::DEBUG::2014-05-13 11:08:32,161::task::595::TaskManager.Task::(_updateState) Task=`484e444f-6639-4616-9b87-efce4a825288`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:08:32,163::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:08:32,164::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00240148', 'lastCheck': '10.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:08:32,164::task::1185::TaskManager.Task::(prepare) Task=`484e444f-6639-4616-9b87-efce4a825288`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00240148', 'lastCheck': '10.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:08:32,165::task::595::TaskManager.Task::(_updateState) Task=`484e444f-6639-4616-9b87-efce4a825288`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:08:32,166::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-322::DEBUG::2014-05-13 11:08:32,166::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000584702 s, 7.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:08:32,167::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:08:32,168::task::990::TaskManager.Task::(_decref) Task=`484e444f-6639-4616-9b87-efce4a825288`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:08:32,353::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:08:32,797::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:08:32,798::task::595::TaskManager.Task::(_updateState) Task=`96706056-184b-4819-8e7b-d0933209079b`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:08:32,798::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:08:32,798::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:08:32,798::task::1185::TaskManager.Task::(prepare) Task=`96706056-184b-4819-8e7b-d0933209079b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:08:32,799::task::595::TaskManager.Task::(_updateState) Task=`96706056-184b-4819-8e7b-d0933209079b`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:08:32,799::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:08:32,799::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:08:32,799::task::990::TaskManager.Task::(_decref) Task=`96706056-184b-4819-8e7b-d0933209079b`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:08:32,804::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:08:32,804::task::595::TaskManager.Task::(_updateState) Task=`ff39d5f2-706d-46de-92a3-e7f02f04c60e`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:08:32,805::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:08:32,805::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`69daef75-9ca0-4fb4-903a-ce5acdb53ffc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:08:32,805::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:08:32,806::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:08:32,806::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`69daef75-9ca0-4fb4-903a-ce5acdb53ffc`::Granted request >Thread-338::DEBUG::2014-05-13 11:08:32,806::task::827::TaskManager.Task::(resourceAcquired) Task=`ff39d5f2-706d-46de-92a3-e7f02f04c60e`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:08:32,806::task::990::TaskManager.Task::(_decref) Task=`ff39d5f2-706d-46de-92a3-e7f02f04c60e`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:08:32,807::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:08:32,807::task::1185::TaskManager.Task::(prepare) Task=`ff39d5f2-706d-46de-92a3-e7f02f04c60e`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:08:32,807::task::595::TaskManager.Task::(_updateState) Task=`ff39d5f2-706d-46de-92a3-e7f02f04c60e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:08:32,808::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:08:32,808::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:08:32,808::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:08:32,808::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:08:32,808::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:08:32,809::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:08:32,809::task::990::TaskManager.Task::(_decref) Task=`ff39d5f2-706d-46de-92a3-e7f02f04c60e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:08:33,636::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:08:33,720::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0377321 s, 27.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:08:35,372::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:08:35,733::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:08:35,814::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374062 s, 27.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:08:37,827::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:08:37,906::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0350178 s, 29.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:08:38,396::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:08:39,919::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:08:39,993::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0300444 s, 34.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:08:41,421::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:08:42,006::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:08:42,085::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353218 s, 29.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:08:42,189::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:08:42,198::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000745766 s, 5.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:08:42,827::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:08:42,828::task::595::TaskManager.Task::(_updateState) Task=`e114e25e-b227-4923-926d-f40c6072a9b8`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:08:42,828::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:08:42,828::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:08:42,828::task::1185::TaskManager.Task::(prepare) Task=`e114e25e-b227-4923-926d-f40c6072a9b8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:08:42,828::task::595::TaskManager.Task::(_updateState) Task=`e114e25e-b227-4923-926d-f40c6072a9b8`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:08:42,829::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:08:42,829::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:08:42,829::task::990::TaskManager.Task::(_decref) Task=`e114e25e-b227-4923-926d-f40c6072a9b8`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:08:42,834::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:08:42,834::task::595::TaskManager.Task::(_updateState) Task=`94e56641-4760-48c1-bca3-afeabb4ec369`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:08:42,834::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:08:42,835::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`59adb1f7-3546-4270-a318-d6473143174b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:08:42,835::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:08:42,835::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:08:42,835::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`59adb1f7-3546-4270-a318-d6473143174b`::Granted request >Thread-338::DEBUG::2014-05-13 11:08:42,836::task::827::TaskManager.Task::(resourceAcquired) Task=`94e56641-4760-48c1-bca3-afeabb4ec369`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:08:42,836::task::990::TaskManager.Task::(_decref) Task=`94e56641-4760-48c1-bca3-afeabb4ec369`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:08:42,837::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:08:42,837::task::1185::TaskManager.Task::(prepare) Task=`94e56641-4760-48c1-bca3-afeabb4ec369`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:08:42,837::task::595::TaskManager.Task::(_updateState) Task=`94e56641-4760-48c1-bca3-afeabb4ec369`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:08:42,837::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:08:42,838::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:08:42,838::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:08:42,838::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:08:42,838::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:08:42,838::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:08:42,839::task::990::TaskManager.Task::(_decref) Task=`94e56641-4760-48c1-bca3-afeabb4ec369`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:08:44,098::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:08:44,172::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306563 s, 33.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:08:44,441::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:08:46,185::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:08:46,557::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.328922 s, 3.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:08:47,319::task::595::TaskManager.Task::(_updateState) Task=`98d27803-0da8-4827-9f9d-de8b322c7240`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:08:47,319::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:08:47,319::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000745766', 'lastCheck': '5.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:08:47,319::task::1185::TaskManager.Task::(prepare) Task=`98d27803-0da8-4827-9f9d-de8b322c7240`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000745766', 'lastCheck': '5.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:08:47,319::task::595::TaskManager.Task::(_updateState) Task=`98d27803-0da8-4827-9f9d-de8b322c7240`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:08:47,320::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:08:47,320::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:08:47,320::task::990::TaskManager.Task::(_decref) Task=`98d27803-0da8-4827-9f9d-de8b322c7240`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:08:47,467::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:08:48,570::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:08:48,655::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.041811 s, 24.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:08:50,486::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:08:50,669::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:08:50,744::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323862 s, 31.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:08:52,204::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:08:52,212::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000680785 s, 6.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:08:52,757::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:08:52,836::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0354348 s, 28.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:08:52,858::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6849800f] >Thread-66::DEBUG::2014-05-13 11:08:52,858::task::595::TaskManager.Task::(_updateState) Task=`f52ae96d-6155-4c2e-bfd3-8a7d92a56890`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:08:52,858::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:08:52,859::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:08:52,859::task::1185::TaskManager.Task::(prepare) Task=`f52ae96d-6155-4c2e-bfd3-8a7d92a56890`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:08:52,859::task::595::TaskManager.Task::(_updateState) Task=`f52ae96d-6155-4c2e-bfd3-8a7d92a56890`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:08:52,859::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:08:52,859::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:08:52,860::task::990::TaskManager.Task::(_decref) Task=`f52ae96d-6155-4c2e-bfd3-8a7d92a56890`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:08:52,864::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6849800f] >Thread-338::DEBUG::2014-05-13 11:08:52,864::task::595::TaskManager.Task::(_updateState) Task=`b69e9e19-f671-4afc-8337-c32c6e855c88`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:08:52,865::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:08:52,865::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2c53d451-3749-4212-af60-ae435e29858b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:08:52,865::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:08:52,865::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:08:52,866::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2c53d451-3749-4212-af60-ae435e29858b`::Granted request >Thread-338::DEBUG::2014-05-13 11:08:52,866::task::827::TaskManager.Task::(resourceAcquired) Task=`b69e9e19-f671-4afc-8337-c32c6e855c88`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:08:52,866::task::990::TaskManager.Task::(_decref) Task=`b69e9e19-f671-4afc-8337-c32c6e855c88`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:08:52,867::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:08:52,867::task::1185::TaskManager.Task::(prepare) Task=`b69e9e19-f671-4afc-8337-c32c6e855c88`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:08:52,867::task::595::TaskManager.Task::(_updateState) Task=`b69e9e19-f671-4afc-8337-c32c6e855c88`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:08:52,868::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:08:52,868::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:08:52,868::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:08:52,868::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:08:52,868::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:08:52,869::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:08:52,869::task::990::TaskManager.Task::(_decref) Task=`b69e9e19-f671-4afc-8337-c32c6e855c88`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:08:53,505::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:08:54,849::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:08:54,927::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0331095 s, 30.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:08:56,523::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:08:56,940::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:08:57,019::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0354482 s, 28.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:08:59,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:08:59,397::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.321203 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:08:59,541::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:09:01,410::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:01,486::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313425 s, 32.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:09:02,244::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:09:02,253::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000675169 s, 6.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:09:02,459::task::595::TaskManager.Task::(_updateState) Task=`53472378-0c48-4fe1-b7f7-33862b3e8d40`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:09:02,459::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:09:02,460::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000675169', 'lastCheck': '0.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:09:02,460::task::1185::TaskManager.Task::(prepare) Task=`53472378-0c48-4fe1-b7f7-33862b3e8d40`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000675169', 'lastCheck': '0.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:09:02,460::task::595::TaskManager.Task::(_updateState) Task=`53472378-0c48-4fe1-b7f7-33862b3e8d40`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:09:02,460::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:09:02,460::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:09:02,460::task::990::TaskManager.Task::(_decref) Task=`53472378-0c48-4fe1-b7f7-33862b3e8d40`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:09:02,563::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:09:02,886::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1c80df6] >Thread-66::DEBUG::2014-05-13 11:09:02,886::task::595::TaskManager.Task::(_updateState) Task=`5f76f8d0-182d-428e-97d4-9d3254b1149c`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:09:02,887::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:09:02,887::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:09:02,887::task::1185::TaskManager.Task::(prepare) Task=`5f76f8d0-182d-428e-97d4-9d3254b1149c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:09:02,887::task::595::TaskManager.Task::(_updateState) Task=`5f76f8d0-182d-428e-97d4-9d3254b1149c`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:09:02,888::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:09:02,888::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:09:02,888::task::990::TaskManager.Task::(_decref) Task=`5f76f8d0-182d-428e-97d4-9d3254b1149c`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:09:02,893::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1c80df6] >Thread-338::DEBUG::2014-05-13 11:09:02,893::task::595::TaskManager.Task::(_updateState) Task=`14090bc9-1b23-462c-8ade-bc181410d85d`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:09:02,894::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:09:02,894::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d17fc444-9875-40e0-aac8-aae38531f431`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:09:02,894::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:09:02,895::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:09:02,895::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d17fc444-9875-40e0-aac8-aae38531f431`::Granted request >Thread-338::DEBUG::2014-05-13 11:09:02,895::task::827::TaskManager.Task::(resourceAcquired) Task=`14090bc9-1b23-462c-8ade-bc181410d85d`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:09:02,895::task::990::TaskManager.Task::(_decref) Task=`14090bc9-1b23-462c-8ade-bc181410d85d`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:09:02,896::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:09:02,896::task::1185::TaskManager.Task::(prepare) Task=`14090bc9-1b23-462c-8ade-bc181410d85d`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:09:02,897::task::595::TaskManager.Task::(_updateState) Task=`14090bc9-1b23-462c-8ade-bc181410d85d`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:09:02,897::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:09:02,897::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:09:02,897::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:09:02,897::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:09:02,898::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:09:02,898::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:09:02,898::task::990::TaskManager.Task::(_decref) Task=`14090bc9-1b23-462c-8ade-bc181410d85d`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:09:03,499::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:03,589::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.047633 s, 21.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:09:05,585::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:09:05,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:05,679::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.032245 s, 31.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:09:07,692::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:07,770::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355603 s, 28.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:09:08,604::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:09:09,783::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:09,863::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362177 s, 28.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:09:11,623::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:09:11,876::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:12,243::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.324465 s, 3.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:09:12,260::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:09:12,268::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000607801 s, 6.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:09:12,915::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17998964] >Thread-66::DEBUG::2014-05-13 11:09:12,915::task::595::TaskManager.Task::(_updateState) Task=`a4558d4b-36a2-4819-b0d4-b25af79f5929`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:09:12,916::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:09:12,916::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:09:12,916::task::1185::TaskManager.Task::(prepare) Task=`a4558d4b-36a2-4819-b0d4-b25af79f5929`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:09:12,916::task::595::TaskManager.Task::(_updateState) Task=`a4558d4b-36a2-4819-b0d4-b25af79f5929`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:09:12,916::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:09:12,917::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:09:12,917::task::990::TaskManager.Task::(_decref) Task=`a4558d4b-36a2-4819-b0d4-b25af79f5929`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:09:12,921::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17998964] >Thread-338::DEBUG::2014-05-13 11:09:12,922::task::595::TaskManager.Task::(_updateState) Task=`475e3c43-3204-4f53-ad91-b87bf1f217a5`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:09:12,922::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:09:12,922::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c50533ac-4fcd-4d8d-b109-dcd2d0a067f3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:09:12,922::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:09:12,923::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:09:12,923::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c50533ac-4fcd-4d8d-b109-dcd2d0a067f3`::Granted request >Thread-338::DEBUG::2014-05-13 11:09:12,923::task::827::TaskManager.Task::(resourceAcquired) Task=`475e3c43-3204-4f53-ad91-b87bf1f217a5`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:09:12,923::task::990::TaskManager.Task::(_decref) Task=`475e3c43-3204-4f53-ad91-b87bf1f217a5`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:09:12,924::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:09:12,924::task::1185::TaskManager.Task::(prepare) Task=`475e3c43-3204-4f53-ad91-b87bf1f217a5`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:09:12,924::task::595::TaskManager.Task::(_updateState) Task=`475e3c43-3204-4f53-ad91-b87bf1f217a5`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:09:12,924::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:09:12,925::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:09:12,925::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:09:12,925::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:09:12,925::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:09:12,925::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:09:12,925::task::990::TaskManager.Task::(_decref) Task=`475e3c43-3204-4f53-ad91-b87bf1f217a5`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:09:14,257::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:14,331::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311942 s, 32.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:09:14,642::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:09:16,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:16,419::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313043 s, 32.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:09:17,595::task::595::TaskManager.Task::(_updateState) Task=`e99af163-5a48-4621-bd41-d6cd76e61be8`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:09:17,595::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:09:17,595::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000607801', 'lastCheck': '5.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:09:17,596::task::1185::TaskManager.Task::(prepare) Task=`e99af163-5a48-4621-bd41-d6cd76e61be8`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000607801', 'lastCheck': '5.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:09:17,596::task::595::TaskManager.Task::(_updateState) Task=`e99af163-5a48-4621-bd41-d6cd76e61be8`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:09:17,596::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:09:17,596::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:09:17,596::task::990::TaskManager.Task::(_decref) Task=`e99af163-5a48-4621-bd41-d6cd76e61be8`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:09:17,661::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:09:18,432::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:18,512::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035273 s, 29.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:09:20,525::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:20,600::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0327495 s, 31.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:09:20,679::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:09:22,275::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:09:22,283::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000724789 s, 5.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:09:22,613::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:22,692::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.034985 s, 29.3 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:09:22,942::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:09:22,943::task::595::TaskManager.Task::(_updateState) Task=`78fc7dff-c5dd-46b1-b140-4fe89b75dfe6`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:09:22,943::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:09:22,943::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:09:22,944::task::1185::TaskManager.Task::(prepare) Task=`78fc7dff-c5dd-46b1-b140-4fe89b75dfe6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:09:22,944::task::595::TaskManager.Task::(_updateState) Task=`78fc7dff-c5dd-46b1-b140-4fe89b75dfe6`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:09:22,944::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:09:22,944::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:09:22,944::task::990::TaskManager.Task::(_decref) Task=`78fc7dff-c5dd-46b1-b140-4fe89b75dfe6`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:09:22,949::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:09:22,949::task::595::TaskManager.Task::(_updateState) Task=`1150036b-050c-422c-993e-a51489920c02`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:09:22,950::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:09:22,950::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8bab8962-d88d-44ed-b6a0-1960cd756327`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:09:22,950::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:09:22,951::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:09:22,951::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8bab8962-d88d-44ed-b6a0-1960cd756327`::Granted request >Thread-338::DEBUG::2014-05-13 11:09:22,951::task::827::TaskManager.Task::(resourceAcquired) Task=`1150036b-050c-422c-993e-a51489920c02`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:09:22,951::task::990::TaskManager.Task::(_decref) Task=`1150036b-050c-422c-993e-a51489920c02`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:09:22,952::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:09:22,952::task::1185::TaskManager.Task::(prepare) Task=`1150036b-050c-422c-993e-a51489920c02`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:09:22,952::task::595::TaskManager.Task::(_updateState) Task=`1150036b-050c-422c-993e-a51489920c02`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:09:22,953::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:09:22,953::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:09:22,953::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:09:22,953::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:09:22,953::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:09:22,953::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:09:22,954::task::990::TaskManager.Task::(_decref) Task=`1150036b-050c-422c-993e-a51489920c02`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:09:23,697::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:09:24,705::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:24,784::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359799 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:09:26,716::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:09:26,797::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:26,886::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0449315 s, 22.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:09:28,899::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:28,975::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312855 s, 32.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:09:29,735::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:09:30,988::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:31,064::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316087 s, 32.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:09:32,290::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:09:32,299::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000676298 s, 6.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:09:32,754::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:09:32,767::task::595::TaskManager.Task::(_updateState) Task=`f7641338-6d17-4027-a2c9-7db62eb6ce59`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:09:32,767::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:09:32,768::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000676298', 'lastCheck': '0.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:09:32,768::task::1185::TaskManager.Task::(prepare) Task=`f7641338-6d17-4027-a2c9-7db62eb6ce59`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000676298', 'lastCheck': '0.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:09:32,768::task::595::TaskManager.Task::(_updateState) Task=`f7641338-6d17-4027-a2c9-7db62eb6ce59`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:09:32,768::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:09:32,768::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:09:32,769::task::990::TaskManager.Task::(_decref) Task=`f7641338-6d17-4027-a2c9-7db62eb6ce59`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 11:09:32,975::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:09:32,976::task::595::TaskManager.Task::(_updateState) Task=`887b21ac-5cc7-4097-96d5-c0aa4a50318e`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:09:32,976::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:09:32,976::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:09:32,976::task::1185::TaskManager.Task::(prepare) Task=`887b21ac-5cc7-4097-96d5-c0aa4a50318e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:09:32,976::task::595::TaskManager.Task::(_updateState) Task=`887b21ac-5cc7-4097-96d5-c0aa4a50318e`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:09:32,977::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:09:32,977::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:09:32,977::task::990::TaskManager.Task::(_decref) Task=`887b21ac-5cc7-4097-96d5-c0aa4a50318e`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:09:32,982::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:09:32,982::task::595::TaskManager.Task::(_updateState) Task=`cfa0f356-adb1-4692-b456-ff23d9476fad`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:09:32,983::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:09:32,984::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8b48382b-8fe6-4638-bb0a-629f785f7397`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:09:32,984::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:09:32,984::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:09:32,984::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8b48382b-8fe6-4638-bb0a-629f785f7397`::Granted request >Thread-338::DEBUG::2014-05-13 11:09:32,985::task::827::TaskManager.Task::(resourceAcquired) Task=`cfa0f356-adb1-4692-b456-ff23d9476fad`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:09:32,985::task::990::TaskManager.Task::(_decref) Task=`cfa0f356-adb1-4692-b456-ff23d9476fad`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:09:32,986::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:09:32,986::task::1185::TaskManager.Task::(prepare) Task=`cfa0f356-adb1-4692-b456-ff23d9476fad`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:09:32,986::task::595::TaskManager.Task::(_updateState) Task=`cfa0f356-adb1-4692-b456-ff23d9476fad`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:09:32,986::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:09:32,986::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:09:32,987::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:09:32,987::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:09:32,987::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:09:32,987::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:09:32,987::task::990::TaskManager.Task::(_decref) Task=`cfa0f356-adb1-4692-b456-ff23d9476fad`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:09:33,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:33,156::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0367174 s, 27.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:09:35,169::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:35,244::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0329227 s, 31.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:09:35,784::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:09:37,257::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:37,332::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309078 s, 33.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:09:38,804::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:09:39,345::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:39,438::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0492978 s, 20.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:09:41,450::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:41,525::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0308606 s, 33.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:09:41,822::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:09:42,306::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:09:42,314::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00072852 s, 5.6 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:09:43,006::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:09:43,006::task::595::TaskManager.Task::(_updateState) Task=`9557d3a3-41f0-4f46-bfce-70ee972a71a6`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:09:43,007::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:09:43,007::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:09:43,007::task::1185::TaskManager.Task::(prepare) Task=`9557d3a3-41f0-4f46-bfce-70ee972a71a6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:09:43,007::task::595::TaskManager.Task::(_updateState) Task=`9557d3a3-41f0-4f46-bfce-70ee972a71a6`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:09:43,007::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:09:43,008::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:09:43,008::task::990::TaskManager.Task::(_decref) Task=`9557d3a3-41f0-4f46-bfce-70ee972a71a6`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:09:43,012::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:09:43,012::task::595::TaskManager.Task::(_updateState) Task=`897a9dc8-1995-4b8b-9afd-75311107d5b3`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:09:43,013::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:09:43,013::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`90915a95-9324-42d7-8290-8b5e0a7dd298`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:09:43,013::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:09:43,013::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:09:43,014::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`90915a95-9324-42d7-8290-8b5e0a7dd298`::Granted request >Thread-338::DEBUG::2014-05-13 11:09:43,014::task::827::TaskManager.Task::(resourceAcquired) Task=`897a9dc8-1995-4b8b-9afd-75311107d5b3`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:09:43,014::task::990::TaskManager.Task::(_decref) Task=`897a9dc8-1995-4b8b-9afd-75311107d5b3`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:09:43,015::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:09:43,015::task::1185::TaskManager.Task::(prepare) Task=`897a9dc8-1995-4b8b-9afd-75311107d5b3`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:09:43,015::task::595::TaskManager.Task::(_updateState) Task=`897a9dc8-1995-4b8b-9afd-75311107d5b3`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:09:43,015::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:09:43,015::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:09:43,016::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:09:43,016::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:09:43,016::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:09:43,016::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:09:43,016::task::990::TaskManager.Task::(_decref) Task=`897a9dc8-1995-4b8b-9afd-75311107d5b3`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:09:43,538::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:43,614::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313073 s, 32.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:09:44,842::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:09:45,627::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:45,705::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364366 s, 28.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:09:47,718::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:47,797::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0351667 s, 29.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:09:47,860::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:09:47,902::task::595::TaskManager.Task::(_updateState) Task=`095a24cc-e8df-4e92-ad3c-a7bef1c53c5c`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:09:47,903::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:09:47,903::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00072852', 'lastCheck': '5.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:09:47,903::task::1185::TaskManager.Task::(prepare) Task=`095a24cc-e8df-4e92-ad3c-a7bef1c53c5c`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00072852', 'lastCheck': '5.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:09:47,903::task::595::TaskManager.Task::(_updateState) Task=`095a24cc-e8df-4e92-ad3c-a7bef1c53c5c`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:09:47,904::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:09:47,904::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:09:47,904::task::990::TaskManager.Task::(_decref) Task=`095a24cc-e8df-4e92-ad3c-a7bef1c53c5c`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:09:49,810::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:49,884::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.030811 s, 33.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:09:50,878::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:09:51,897::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:51,986::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0460317 s, 22.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:09:52,319::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 41830dad-f012-4b4d-b636-863f66d5ca4b >Thread-322::DEBUG::2014-05-13 11:09:52,319::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-322::DEBUG::2014-05-13 11:09:52,322::blockSD::336::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:09:52,330::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000661775 s, 3.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:09:52,331::misc::287::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000661775 s, 3.1 MB/s'], size: 2048 >Thread-322::DEBUG::2014-05-13 11:09:52,331::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-322::WARNING::2014-05-13 11:09:52,331::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-322::DEBUG::2014-05-13 11:09:52,331::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-322::DEBUG::2014-05-13 11:09:52,331::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:09:52,332::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:09:52,332::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:09:52,332::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:09:52,333::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:09:52,333::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 11:09:52,689::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:09:52,691::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:09:52,691::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'LOCKPOLICY=', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', 'DESCRIPTION=ISCSI', 'LOGBLKSIZE=512', 'TYPE=ISCSI', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'PHYBLKSIZE=512', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', 'CLASS=Data', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'POOL_DESCRIPTION=DC_2', 'POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'MASTER_VERSION=1', 'ROLE=Master', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', '_SHA_CKSUM=28bb2d7cc60e0d779eb69fe93d050c16c769f069', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0'] >Thread-322::INFO::2014-05-13 11:09:52,692::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_imageNS already registered >Thread-322::INFO::2014-05-13 11:09:52,692::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_volumeNS already registered >Thread-322::INFO::2014-05-13 11:09:52,693::blockSD::452::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_lvmActivationNS already registered >Thread-322::DEBUG::2014-05-13 11:09:52,693::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 11:09:52,876::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:09:52,880::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:09:52,898::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00989396 s, 414 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:09:53,036::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:09:53,036::task::595::TaskManager.Task::(_updateState) Task=`ca2a6f3d-5c39-46e0-81a1-1bba8fdfc085`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:09:53,037::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:09:53,037::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:09:53,037::task::1185::TaskManager.Task::(prepare) Task=`ca2a6f3d-5c39-46e0-81a1-1bba8fdfc085`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:09:53,037::task::595::TaskManager.Task::(_updateState) Task=`ca2a6f3d-5c39-46e0-81a1-1bba8fdfc085`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:09:53,037::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:09:53,038::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:09:53,038::task::990::TaskManager.Task::(_decref) Task=`ca2a6f3d-5c39-46e0-81a1-1bba8fdfc085`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:09:53,042::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:09:53,043::task::595::TaskManager.Task::(_updateState) Task=`31076c71-23b1-4e37-b7fd-dbce3ada0f57`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:09:53,043::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:09:53,043::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a25f2c8a-dce4-4072-a3ef-b08ae72c9ab6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:09:53,044::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:09:53,044::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:09:53,044::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a25f2c8a-dce4-4072-a3ef-b08ae72c9ab6`::Granted request >Thread-338::DEBUG::2014-05-13 11:09:53,045::task::827::TaskManager.Task::(resourceAcquired) Task=`31076c71-23b1-4e37-b7fd-dbce3ada0f57`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:09:53,045::task::990::TaskManager.Task::(_decref) Task=`31076c71-23b1-4e37-b7fd-dbce3ada0f57`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:09:53,045::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:09:53,046::task::1185::TaskManager.Task::(prepare) Task=`31076c71-23b1-4e37-b7fd-dbce3ada0f57`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:09:53,046::task::595::TaskManager.Task::(_updateState) Task=`31076c71-23b1-4e37-b7fd-dbce3ada0f57`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:09:53,046::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:09:53,046::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:09:53,046::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:09:53,047::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:09:53,047::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:09:53,047::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:09:53,047::task::990::TaskManager.Task::(_decref) Task=`31076c71-23b1-4e37-b7fd-dbce3ada0f57`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:09:53,898::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:09:53,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:54,079::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364744 s, 28.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:09:56,091::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:56,166::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311053 s, 32.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:09:56,918::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:09:58,179::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:09:58,253::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303726 s, 33.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:09:59,936::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:10:00,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:00,346::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036509 s, 28.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:10:02,358::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:02,433::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031127 s, 32.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:10:02,905::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:10:02,914::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000710377 s, 5.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:10:02,954::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:10:03,048::task::595::TaskManager.Task::(_updateState) Task=`173d4b61-335e-46ee-b4c7-25517e0ef419`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:10:03,048::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:10:03,049::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000710377', 'lastCheck': '0.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:10:03,049::task::1185::TaskManager.Task::(prepare) Task=`173d4b61-335e-46ee-b4c7-25517e0ef419`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000710377', 'lastCheck': '0.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:10:03,049::task::595::TaskManager.Task::(_updateState) Task=`173d4b61-335e-46ee-b4c7-25517e0ef419`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:10:03,049::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:10:03,049::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:10:03,050::task::990::TaskManager.Task::(_decref) Task=`173d4b61-335e-46ee-b4c7-25517e0ef419`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 11:10:03,065::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:10:03,065::task::595::TaskManager.Task::(_updateState) Task=`cb7a7230-8310-4edb-b21e-c6489ee1271e`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:10:03,066::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:10:03,066::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:10:03,066::task::1185::TaskManager.Task::(prepare) Task=`cb7a7230-8310-4edb-b21e-c6489ee1271e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:10:03,066::task::595::TaskManager.Task::(_updateState) Task=`cb7a7230-8310-4edb-b21e-c6489ee1271e`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:10:03,066::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:10:03,066::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:10:03,067::task::990::TaskManager.Task::(_decref) Task=`cb7a7230-8310-4edb-b21e-c6489ee1271e`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:10:03,111::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:10:03,112::task::595::TaskManager.Task::(_updateState) Task=`e5ec1a60-fc0e-4fc0-8493-82799737205c`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:10:03,112::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:10:03,112::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f89126d7-a045-4eb9-ac0c-ce6784f5d6dc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:10:03,113::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:10:03,113::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:10:03,113::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f89126d7-a045-4eb9-ac0c-ce6784f5d6dc`::Granted request >Thread-338::DEBUG::2014-05-13 11:10:03,114::task::827::TaskManager.Task::(resourceAcquired) Task=`e5ec1a60-fc0e-4fc0-8493-82799737205c`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:10:03,114::task::990::TaskManager.Task::(_decref) Task=`e5ec1a60-fc0e-4fc0-8493-82799737205c`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:10:03,115::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:10:03,115::task::1185::TaskManager.Task::(prepare) Task=`e5ec1a60-fc0e-4fc0-8493-82799737205c`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:10:03,115::task::595::TaskManager.Task::(_updateState) Task=`e5ec1a60-fc0e-4fc0-8493-82799737205c`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:10:03,116::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:10:03,116::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:10:03,116::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:10:03,116::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:10:03,116::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:10:03,117::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:10:03,117::task::990::TaskManager.Task::(_decref) Task=`e5ec1a60-fc0e-4fc0-8493-82799737205c`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:10:04,446::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:04,522::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312394 s, 32.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:10:05,971::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:10:06,535::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:06,895::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.317277 s, 3.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:10:08,909::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:08,989::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359901 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:10:09,002::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:10:11,002::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:11,075::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303918 s, 33.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:10:12,021::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:10:12,922::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:10:12,930::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000646638 s, 6.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:10:13,088::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 11:10:13,137::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [7a000b17] >Thread-66::DEBUG::2014-05-13 11:10:13,139::task::595::TaskManager.Task::(_updateState) Task=`421e90e0-bdb7-471d-b3e6-06f59ccc3aba`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:10:13,141::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:10:13,142::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:10:13,143::task::1185::TaskManager.Task::(prepare) Task=`421e90e0-bdb7-471d-b3e6-06f59ccc3aba`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:10:13,144::task::595::TaskManager.Task::(_updateState) Task=`421e90e0-bdb7-471d-b3e6-06f59ccc3aba`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:10:13,145::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:10:13,146::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:10:13,147::task::990::TaskManager.Task::(_decref) Task=`421e90e0-bdb7-471d-b3e6-06f59ccc3aba`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:10:13,155::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [7a000b17] >Thread-338::DEBUG::2014-05-13 11:10:13,156::task::595::TaskManager.Task::(_updateState) Task=`f5a59d38-3e49-490d-a554-29c30626ed8e`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:10:13,157::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:10:13,160::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a0c8f171-ef66-4804-9ea8-5861833340db`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:10:13,160::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:10:13,162::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:10:13,163::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a0c8f171-ef66-4804-9ea8-5861833340db`::Granted request >Thread-338::DEBUG::2014-05-13 11:10:13,164::task::827::TaskManager.Task::(resourceAcquired) Task=`f5a59d38-3e49-490d-a554-29c30626ed8e`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:10:13,165::task::990::TaskManager.Task::(_decref) Task=`f5a59d38-3e49-490d-a554-29c30626ed8e`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:10:13,170::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:10:13,170::task::1185::TaskManager.Task::(prepare) Task=`f5a59d38-3e49-490d-a554-29c30626ed8e`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:10:13,171::task::595::TaskManager.Task::(_updateState) Task=`f5a59d38-3e49-490d-a554-29c30626ed8e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:10:13,172::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:10:13,173::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:10:13,175::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:10:13,176::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:10:13,177::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:10:13,177::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:10:13,179::task::990::TaskManager.Task::(_decref) Task=`f5a59d38-3e49-490d-a554-29c30626ed8e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:10:13,204::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319332 s, 32.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:10:15,038::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:10:15,216::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:15,298::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387911 s, 26.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:10:17,311::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:17,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317979 s, 32.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:10:18,056::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:10:18,153::task::595::TaskManager.Task::(_updateState) Task=`c11c0662-206b-4349-ada1-38d386d69a23`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:10:18,154::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:10:18,154::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000646638', 'lastCheck': '5.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:10:18,154::task::1185::TaskManager.Task::(prepare) Task=`c11c0662-206b-4349-ada1-38d386d69a23`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000646638', 'lastCheck': '5.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:10:18,154::task::595::TaskManager.Task::(_updateState) Task=`c11c0662-206b-4349-ada1-38d386d69a23`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:10:18,154::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:10:18,155::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:10:18,155::task::990::TaskManager.Task::(_decref) Task=`c11c0662-206b-4349-ada1-38d386d69a23`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:10:19,400::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:19,490::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0462279 s, 22.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:10:21,075::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:10:21,503::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:21,584::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365039 s, 28.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:10:22,938::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:10:22,947::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000762303 s, 5.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:10:23,199::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:10:23,200::task::595::TaskManager.Task::(_updateState) Task=`388d50cc-0816-4397-9611-2c14956fc366`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:10:23,200::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:10:23,200::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:10:23,201::task::1185::TaskManager.Task::(prepare) Task=`388d50cc-0816-4397-9611-2c14956fc366`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:10:23,201::task::595::TaskManager.Task::(_updateState) Task=`388d50cc-0816-4397-9611-2c14956fc366`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:10:23,201::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:10:23,201::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:10:23,201::task::990::TaskManager.Task::(_decref) Task=`388d50cc-0816-4397-9611-2c14956fc366`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:10:23,206::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:10:23,206::task::595::TaskManager.Task::(_updateState) Task=`4f809096-9ef2-4f01-85b2-8e1e26ae84fc`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:10:23,207::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:10:23,207::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1577e507-bc6f-40c8-8a1c-afb5b8ab4f0c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:10:23,207::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:10:23,207::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:10:23,208::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1577e507-bc6f-40c8-8a1c-afb5b8ab4f0c`::Granted request >Thread-338::DEBUG::2014-05-13 11:10:23,208::task::827::TaskManager.Task::(resourceAcquired) Task=`4f809096-9ef2-4f01-85b2-8e1e26ae84fc`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:10:23,208::task::990::TaskManager.Task::(_decref) Task=`4f809096-9ef2-4f01-85b2-8e1e26ae84fc`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:10:23,209::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:10:23,209::task::1185::TaskManager.Task::(prepare) Task=`4f809096-9ef2-4f01-85b2-8e1e26ae84fc`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:10:23,209::task::595::TaskManager.Task::(_updateState) Task=`4f809096-9ef2-4f01-85b2-8e1e26ae84fc`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:10:23,209::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:10:23,210::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:10:23,210::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:10:23,210::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:10:23,210::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:10:23,210::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:10:23,211::task::990::TaskManager.Task::(_decref) Task=`4f809096-9ef2-4f01-85b2-8e1e26ae84fc`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:10:23,597::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:23,683::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0430343 s, 23.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:10:24,098::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:10:25,696::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:25,789::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312407 s, 32.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:10:27,120::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:10:27,802::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:27,883::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361556 s, 28.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:10:29,895::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:29,970::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312229 s, 32.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:10:30,146::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:10:31,983::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:32,355::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.328872 s, 3.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:10:32,954::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:10:32,963::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000818107 s, 5.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:10:33,163::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:10:33,231::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:10:33,232::task::595::TaskManager.Task::(_updateState) Task=`c47f220a-5de8-4e00-868f-7468e8665e4b`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:10:33,232::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:10:33,232::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:10:33,232::task::1185::TaskManager.Task::(prepare) Task=`c47f220a-5de8-4e00-868f-7468e8665e4b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:10:33,233::task::595::TaskManager.Task::(_updateState) Task=`c47f220a-5de8-4e00-868f-7468e8665e4b`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:10:33,233::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:10:33,233::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:10:33,233::task::990::TaskManager.Task::(_decref) Task=`c47f220a-5de8-4e00-868f-7468e8665e4b`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:10:33,240::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:10:33,240::task::595::TaskManager.Task::(_updateState) Task=`6a50521f-f579-46d2-b269-dcc4c3e9c9c6`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:10:33,240::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:10:33,241::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c955df6a-64ce-4256-8d7c-1f8d7ac07aea`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:10:33,241::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:10:33,241::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:10:33,241::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c955df6a-64ce-4256-8d7c-1f8d7ac07aea`::Granted request >Thread-338::DEBUG::2014-05-13 11:10:33,242::task::827::TaskManager.Task::(resourceAcquired) Task=`6a50521f-f579-46d2-b269-dcc4c3e9c9c6`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:10:33,242::task::990::TaskManager.Task::(_decref) Task=`6a50521f-f579-46d2-b269-dcc4c3e9c9c6`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:10:33,243::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:10:33,243::task::1185::TaskManager.Task::(prepare) Task=`6a50521f-f579-46d2-b269-dcc4c3e9c9c6`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:10:33,243::task::595::TaskManager.Task::(_updateState) Task=`6a50521f-f579-46d2-b269-dcc4c3e9c9c6`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:10:33,243::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:10:33,243::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:10:33,244::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:10:33,244::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:10:33,244::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:10:33,244::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:10:33,245::task::990::TaskManager.Task::(_decref) Task=`6a50521f-f579-46d2-b269-dcc4c3e9c9c6`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 11:10:33,289::task::595::TaskManager.Task::(_updateState) Task=`3a0ff8ed-7e48-4377-8c9d-961958fb77f7`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:10:33,289::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:10:33,289::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000818107', 'lastCheck': '0.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:10:33,289::task::1185::TaskManager.Task::(prepare) Task=`3a0ff8ed-7e48-4377-8c9d-961958fb77f7`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000818107', 'lastCheck': '0.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:10:33,289::task::595::TaskManager.Task::(_updateState) Task=`3a0ff8ed-7e48-4377-8c9d-961958fb77f7`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:10:33,290::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:10:33,290::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:10:33,290::task::990::TaskManager.Task::(_decref) Task=`3a0ff8ed-7e48-4377-8c9d-961958fb77f7`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:10:34,368::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:34,445::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312293 s, 32.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:10:36,182::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:10:36,458::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:36,540::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360182 s, 28.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:10:38,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:38,634::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0375789 s, 27.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:10:39,201::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:10:40,647::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:40,724::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316239 s, 32.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:10:42,219::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:10:42,737::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:42,816::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035411 s, 28.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:10:42,970::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:10:42,978::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000686953 s, 6.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:10:43,264::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:10:43,264::task::595::TaskManager.Task::(_updateState) Task=`4408f1d7-ffa8-4c1e-9d25-a7eafbe31e98`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:10:43,265::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:10:43,265::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:10:43,265::task::1185::TaskManager.Task::(prepare) Task=`4408f1d7-ffa8-4c1e-9d25-a7eafbe31e98`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:10:43,265::task::595::TaskManager.Task::(_updateState) Task=`4408f1d7-ffa8-4c1e-9d25-a7eafbe31e98`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:10:43,265::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:10:43,266::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:10:43,266::task::990::TaskManager.Task::(_decref) Task=`4408f1d7-ffa8-4c1e-9d25-a7eafbe31e98`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:10:43,270::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:10:43,270::task::595::TaskManager.Task::(_updateState) Task=`0848b592-f281-477a-a38a-458f2973d5c9`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:10:43,270::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:10:43,271::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a3d63c6d-195b-4b2f-95be-8467f6eb3153`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:10:43,271::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:10:43,271::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:10:43,271::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a3d63c6d-195b-4b2f-95be-8467f6eb3153`::Granted request >Thread-338::DEBUG::2014-05-13 11:10:43,271::task::827::TaskManager.Task::(resourceAcquired) Task=`0848b592-f281-477a-a38a-458f2973d5c9`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:10:43,272::task::990::TaskManager.Task::(_decref) Task=`0848b592-f281-477a-a38a-458f2973d5c9`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:10:43,272::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:10:43,272::task::1185::TaskManager.Task::(prepare) Task=`0848b592-f281-477a-a38a-458f2973d5c9`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:10:43,273::task::595::TaskManager.Task::(_updateState) Task=`0848b592-f281-477a-a38a-458f2973d5c9`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:10:43,273::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:10:43,273::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:10:43,273::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:10:43,273::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:10:43,273::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:10:43,273::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:10:43,274::task::990::TaskManager.Task::(_decref) Task=`0848b592-f281-477a-a38a-458f2973d5c9`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:10:44,829::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:45,234::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.361448 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:10:45,236::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:10:47,249::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:47,326::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319296 s, 32.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:10:48,259::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:10:48,419::task::595::TaskManager.Task::(_updateState) Task=`813d141b-f55d-4096-972a-bf6bc6bb97f6`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:10:48,419::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:10:48,419::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000686953', 'lastCheck': '5.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:10:48,419::task::1185::TaskManager.Task::(prepare) Task=`813d141b-f55d-4096-972a-bf6bc6bb97f6`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000686953', 'lastCheck': '5.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:10:48,420::task::595::TaskManager.Task::(_updateState) Task=`813d141b-f55d-4096-972a-bf6bc6bb97f6`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:10:48,420::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:10:48,420::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:10:48,420::task::990::TaskManager.Task::(_decref) Task=`813d141b-f55d-4096-972a-bf6bc6bb97f6`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:10:49,339::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:49,414::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307846 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:10:51,278::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:10:51,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:51,509::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355412 s, 28.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:10:52,986::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:10:52,994::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000766469 s, 5.3 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:10:53,292::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:10:53,292::task::595::TaskManager.Task::(_updateState) Task=`cf8c32bf-b0b4-40af-ad0c-af3b3a92e683`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:10:53,292::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:10:53,293::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:10:53,293::task::1185::TaskManager.Task::(prepare) Task=`cf8c32bf-b0b4-40af-ad0c-af3b3a92e683`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:10:53,293::task::595::TaskManager.Task::(_updateState) Task=`cf8c32bf-b0b4-40af-ad0c-af3b3a92e683`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:10:53,293::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:10:53,293::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:10:53,293::task::990::TaskManager.Task::(_decref) Task=`cf8c32bf-b0b4-40af-ad0c-af3b3a92e683`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:10:53,298::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:10:53,298::task::595::TaskManager.Task::(_updateState) Task=`32826756-428a-4ad7-9413-56c19105d21f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:10:53,299::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:10:53,299::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`10b80fc0-a0a9-40a5-a6d8-cfd5da932bb9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:10:53,299::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:10:53,300::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:10:53,300::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`10b80fc0-a0a9-40a5-a6d8-cfd5da932bb9`::Granted request >Thread-338::DEBUG::2014-05-13 11:10:53,300::task::827::TaskManager.Task::(resourceAcquired) Task=`32826756-428a-4ad7-9413-56c19105d21f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:10:53,300::task::990::TaskManager.Task::(_decref) Task=`32826756-428a-4ad7-9413-56c19105d21f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:10:53,301::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:10:53,301::task::1185::TaskManager.Task::(prepare) Task=`32826756-428a-4ad7-9413-56c19105d21f`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:10:53,301::task::595::TaskManager.Task::(_updateState) Task=`32826756-428a-4ad7-9413-56c19105d21f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:10:53,302::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:10:53,302::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:10:53,302::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:10:53,302::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:10:53,302::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:10:53,302::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:10:53,303::task::990::TaskManager.Task::(_decref) Task=`32826756-428a-4ad7-9413-56c19105d21f`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:10:53,524::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:53,605::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360723 s, 28.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:10:54,296::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:10:55,618::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:55,694::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313201 s, 32.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:10:57,315::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:10:57,707::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:57,815::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.063277 s, 16.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:10:59,829::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:10:59,903::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0308427 s, 33.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:11:00,338::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:11:01,916::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:11:01,990::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306992 s, 33.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:11:03,002::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:11:03,010::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000725596 s, 5.6 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:11:03,321::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:11:03,322::task::595::TaskManager.Task::(_updateState) Task=`a25b9e85-f367-4269-88a9-1ee7a0c3181c`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:11:03,322::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:11:03,322::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:11:03,323::task::1185::TaskManager.Task::(prepare) Task=`a25b9e85-f367-4269-88a9-1ee7a0c3181c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:11:03,323::task::595::TaskManager.Task::(_updateState) Task=`a25b9e85-f367-4269-88a9-1ee7a0c3181c`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:11:03,323::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:11:03,323::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:11:03,323::task::990::TaskManager.Task::(_decref) Task=`a25b9e85-f367-4269-88a9-1ee7a0c3181c`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:11:03,328::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:11:03,328::task::595::TaskManager.Task::(_updateState) Task=`f91f678c-7f23-4f73-b786-df326e8b2072`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:11:03,329::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:11:03,329::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b6b51eee-e79c-4b8c-8318-98ec359e91d3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:11:03,329::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:11:03,330::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:11:03,330::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b6b51eee-e79c-4b8c-8318-98ec359e91d3`::Granted request >Thread-338::DEBUG::2014-05-13 11:11:03,330::task::827::TaskManager.Task::(resourceAcquired) Task=`f91f678c-7f23-4f73-b786-df326e8b2072`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:11:03,330::task::990::TaskManager.Task::(_decref) Task=`f91f678c-7f23-4f73-b786-df326e8b2072`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:11:03,331::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:11:03,331::task::1185::TaskManager.Task::(prepare) Task=`f91f678c-7f23-4f73-b786-df326e8b2072`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:11:03,331::task::595::TaskManager.Task::(_updateState) Task=`f91f678c-7f23-4f73-b786-df326e8b2072`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:11:03,332::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:11:03,332::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:11:03,332::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:11:03,332::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:11:03,332::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:11:03,332::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:11:03,333::task::990::TaskManager.Task::(_decref) Task=`f91f678c-7f23-4f73-b786-df326e8b2072`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:11:03,358::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:11:03,553::task::595::TaskManager.Task::(_updateState) Task=`e5c4c36d-a02d-450a-9682-1c718595e340`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:11:03,554::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:11:03,554::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000725596', 'lastCheck': '0.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:11:03,554::task::1185::TaskManager.Task::(prepare) Task=`e5c4c36d-a02d-450a-9682-1c718595e340`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000725596', 'lastCheck': '0.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:11:03,554::task::595::TaskManager.Task::(_updateState) Task=`e5c4c36d-a02d-450a-9682-1c718595e340`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:11:03,554::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:11:03,555::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:11:03,555::task::990::TaskManager.Task::(_decref) Task=`e5c4c36d-a02d-450a-9682-1c718595e340`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:11:04,003::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:11:04,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0304823 s, 33.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:11:06,090::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:11:06,170::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0375221 s, 27.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:11:06,385::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:11:08,183::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:11:08,262::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0371785 s, 27.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:11:09,403::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:11:10,275::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:11:10,350::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323637 s, 31.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:11:12,363::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:11:12,422::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:11:12,459::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0490029 s, 20.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:11:13,017::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:11:13,026::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000668686 s, 6.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:11:13,354::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:11:13,400::task::595::TaskManager.Task::(_updateState) Task=`40c4e644-0535-42ff-977f-cdb04271aef1`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:11:13,400::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:11:13,401::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:11:13,401::task::1185::TaskManager.Task::(prepare) Task=`40c4e644-0535-42ff-977f-cdb04271aef1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:11:13,401::task::595::TaskManager.Task::(_updateState) Task=`40c4e644-0535-42ff-977f-cdb04271aef1`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:11:13,401::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:11:13,402::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:11:13,402::task::990::TaskManager.Task::(_decref) Task=`40c4e644-0535-42ff-977f-cdb04271aef1`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:11:13,406::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:11:13,406::task::595::TaskManager.Task::(_updateState) Task=`9465ccde-fc32-4f3c-9989-8a7a8ee684f6`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:11:13,407::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:11:13,407::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d7173bad-d372-41e7-813d-8f92e911854e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:11:13,407::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:11:13,408::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:11:13,409::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d7173bad-d372-41e7-813d-8f92e911854e`::Granted request >Thread-338::DEBUG::2014-05-13 11:11:13,409::task::827::TaskManager.Task::(resourceAcquired) Task=`9465ccde-fc32-4f3c-9989-8a7a8ee684f6`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:11:13,409::task::990::TaskManager.Task::(_decref) Task=`9465ccde-fc32-4f3c-9989-8a7a8ee684f6`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:11:13,410::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:11:13,410::task::1185::TaskManager.Task::(prepare) Task=`9465ccde-fc32-4f3c-9989-8a7a8ee684f6`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:11:13,410::task::595::TaskManager.Task::(_updateState) Task=`9465ccde-fc32-4f3c-9989-8a7a8ee684f6`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:11:13,411::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:11:13,411::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:11:13,411::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:11:13,411::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:11:13,411::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:11:13,411::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:11:13,412::task::990::TaskManager.Task::(_decref) Task=`9465ccde-fc32-4f3c-9989-8a7a8ee684f6`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:11:14,472::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:11:14,552::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379237 s, 27.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:11:15,450::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:11:16,565::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:11:16,640::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320109 s, 32.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:11:18,470::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:11:18,653::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 11:11:18,703::task::595::TaskManager.Task::(_updateState) Task=`893c1d5a-4b0f-41c1-8db2-2d769221aba1`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:11:18,705::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:11:18,706::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000668686', 'lastCheck': '5.7', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:11:18,707::task::1185::TaskManager.Task::(prepare) Task=`893c1d5a-4b0f-41c1-8db2-2d769221aba1`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000668686', 'lastCheck': '5.7', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:11:18,708::task::595::TaskManager.Task::(_updateState) Task=`893c1d5a-4b0f-41c1-8db2-2d769221aba1`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:11:18,709::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:11:18,709::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:11:18,710::task::990::TaskManager.Task::(_decref) Task=`893c1d5a-4b0f-41c1-8db2-2d769221aba1`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:11:18,761::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036229 s, 28.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:11:20,774::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:11:20,848::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311736 s, 32.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:11:21,494::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:11:22,861::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:11:22,937::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0332927 s, 30.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:11:23,035::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:11:23,043::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000644158 s, 6.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:11:23,432::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6602059] >Thread-66::DEBUG::2014-05-13 11:11:23,432::task::595::TaskManager.Task::(_updateState) Task=`61c8f3d1-ee99-4de2-9685-1f502baef1b8`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:11:23,432::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:11:23,433::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:11:23,433::task::1185::TaskManager.Task::(prepare) Task=`61c8f3d1-ee99-4de2-9685-1f502baef1b8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:11:23,433::task::595::TaskManager.Task::(_updateState) Task=`61c8f3d1-ee99-4de2-9685-1f502baef1b8`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:11:23,433::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:11:23,433::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:11:23,434::task::990::TaskManager.Task::(_decref) Task=`61c8f3d1-ee99-4de2-9685-1f502baef1b8`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:11:23,439::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6602059] >Thread-338::DEBUG::2014-05-13 11:11:23,439::task::595::TaskManager.Task::(_updateState) Task=`1e9df95e-05c9-4ce6-b33c-454e603d504c`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:11:23,440::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:11:23,440::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b7cd45c2-75af-4fe6-adbc-2a6e709bb84f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:11:23,440::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:11:23,440::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:11:23,441::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b7cd45c2-75af-4fe6-adbc-2a6e709bb84f`::Granted request >Thread-338::DEBUG::2014-05-13 11:11:23,441::task::827::TaskManager.Task::(resourceAcquired) Task=`1e9df95e-05c9-4ce6-b33c-454e603d504c`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:11:23,441::task::990::TaskManager.Task::(_decref) Task=`1e9df95e-05c9-4ce6-b33c-454e603d504c`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:11:23,442::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:11:23,442::task::1185::TaskManager.Task::(prepare) Task=`1e9df95e-05c9-4ce6-b33c-454e603d504c`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:11:23,442::task::595::TaskManager.Task::(_updateState) Task=`1e9df95e-05c9-4ce6-b33c-454e603d504c`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:11:23,442::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:11:23,443::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:11:23,443::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:11:23,443::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:11:23,443::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:11:23,443::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:11:23,444::task::990::TaskManager.Task::(_decref) Task=`1e9df95e-05c9-4ce6-b33c-454e603d504c`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:11:24,512::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:11:24,950::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:11:25,034::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382876 s, 26.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:11:27,044::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:11:27,452::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.365712 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:11:27,530::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:11:29,465::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:11:29,544::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356449 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:11:30,549::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:11:31,557::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:11:31,632::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031606 s, 32.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:11:33,051::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:11:33,065::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000595362 s, 6.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:11:33,464::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:11:33,464::task::595::TaskManager.Task::(_updateState) Task=`2ddf9f74-d36b-42bd-9db8-bfc34d5eb485`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:11:33,464::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:11:33,465::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:11:33,465::task::1185::TaskManager.Task::(prepare) Task=`2ddf9f74-d36b-42bd-9db8-bfc34d5eb485`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:11:33,465::task::595::TaskManager.Task::(_updateState) Task=`2ddf9f74-d36b-42bd-9db8-bfc34d5eb485`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:11:33,465::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:11:33,465::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:11:33,465::task::990::TaskManager.Task::(_decref) Task=`2ddf9f74-d36b-42bd-9db8-bfc34d5eb485`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:11:33,470::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:11:33,470::task::595::TaskManager.Task::(_updateState) Task=`acd96f43-db05-4463-afd5-10d2a3c0af6c`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:11:33,470::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:11:33,471::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6ccf2299-448c-456f-967c-1ea3479c35b4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:11:33,471::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:11:33,471::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:11:33,471::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6ccf2299-448c-456f-967c-1ea3479c35b4`::Granted request >Thread-338::DEBUG::2014-05-13 11:11:33,472::task::827::TaskManager.Task::(resourceAcquired) Task=`acd96f43-db05-4463-afd5-10d2a3c0af6c`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:11:33,472::task::990::TaskManager.Task::(_decref) Task=`acd96f43-db05-4463-afd5-10d2a3c0af6c`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:11:33,473::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:11:33,473::task::1185::TaskManager.Task::(prepare) Task=`acd96f43-db05-4463-afd5-10d2a3c0af6c`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:11:33,473::task::595::TaskManager.Task::(_updateState) Task=`acd96f43-db05-4463-afd5-10d2a3c0af6c`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:11:33,473::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:11:33,473::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:11:33,474::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:11:33,474::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:11:33,474::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:11:33,474::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:11:33,474::task::990::TaskManager.Task::(_decref) Task=`acd96f43-db05-4463-afd5-10d2a3c0af6c`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:11:33,578::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:11:33,645::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:11:33,728::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0385085 s, 26.6 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:11:33,862::task::595::TaskManager.Task::(_updateState) Task=`b190abf7-7e1e-44cc-b9b0-8f90ddb97526`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:11:33,863::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:11:33,863::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000595362', 'lastCheck': '0.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:11:33,863::task::1185::TaskManager.Task::(prepare) Task=`b190abf7-7e1e-44cc-b9b0-8f90ddb97526`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000595362', 'lastCheck': '0.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:11:33,863::task::595::TaskManager.Task::(_updateState) Task=`b190abf7-7e1e-44cc-b9b0-8f90ddb97526`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:11:33,863::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:11:33,864::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:11:33,864::task::990::TaskManager.Task::(_decref) Task=`b190abf7-7e1e-44cc-b9b0-8f90ddb97526`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:11:35,740::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:11:35,814::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312158 s, 32.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:11:36,603::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:11:37,827::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:11:37,919::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0484975 s, 21.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:11:39,627::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:11:39,932::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:11:40,030::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0539548 s, 19.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:11:42,043::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:11:42,116::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0302904 s, 33.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:11:42,646::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:11:43,073::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:11:43,081::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000537259 s, 7.6 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:11:43,492::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:11:43,493::task::595::TaskManager.Task::(_updateState) Task=`8f2469c3-049f-4614-bbc2-d0d62a784366`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:11:43,493::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:11:43,494::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:11:43,494::task::1185::TaskManager.Task::(prepare) Task=`8f2469c3-049f-4614-bbc2-d0d62a784366`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:11:43,494::task::595::TaskManager.Task::(_updateState) Task=`8f2469c3-049f-4614-bbc2-d0d62a784366`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:11:43,494::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:11:43,494::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:11:43,495::task::990::TaskManager.Task::(_decref) Task=`8f2469c3-049f-4614-bbc2-d0d62a784366`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:11:43,499::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:11:43,499::task::595::TaskManager.Task::(_updateState) Task=`cfcff8ad-878b-4f65-bb4f-1026a1f79e6e`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:11:43,500::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:11:43,500::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ab414f24-2bca-4c9d-a343-242334747f7a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:11:43,500::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:11:43,501::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:11:43,501::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ab414f24-2bca-4c9d-a343-242334747f7a`::Granted request >Thread-338::DEBUG::2014-05-13 11:11:43,501::task::827::TaskManager.Task::(resourceAcquired) Task=`cfcff8ad-878b-4f65-bb4f-1026a1f79e6e`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:11:43,501::task::990::TaskManager.Task::(_decref) Task=`cfcff8ad-878b-4f65-bb4f-1026a1f79e6e`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:11:43,502::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:11:43,502::task::1185::TaskManager.Task::(prepare) Task=`cfcff8ad-878b-4f65-bb4f-1026a1f79e6e`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:11:43,502::task::595::TaskManager.Task::(_updateState) Task=`cfcff8ad-878b-4f65-bb4f-1026a1f79e6e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:11:43,503::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:11:43,503::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:11:43,503::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:11:43,503::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:11:43,503::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:11:43,504::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:11:43,504::task::990::TaskManager.Task::(_decref) Task=`cfcff8ad-878b-4f65-bb4f-1026a1f79e6e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:11:44,130::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:11:44,209::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037418 s, 27.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:11:45,666::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:11:46,222::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:11:46,296::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309118 s, 33.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:11:48,309::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:11:48,390::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036255 s, 28.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:11:48,686::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:11:49,010::task::595::TaskManager.Task::(_updateState) Task=`a98fa991-05e3-4828-9872-101905e967a6`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:11:49,010::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:11:49,010::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000537259', 'lastCheck': '5.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:11:49,011::task::1185::TaskManager.Task::(prepare) Task=`a98fa991-05e3-4828-9872-101905e967a6`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000537259', 'lastCheck': '5.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:11:49,011::task::595::TaskManager.Task::(_updateState) Task=`a98fa991-05e3-4828-9872-101905e967a6`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:11:49,011::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:11:49,011::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:11:49,011::task::990::TaskManager.Task::(_decref) Task=`a98fa991-05e3-4828-9872-101905e967a6`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:11:50,403::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:11:50,478::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0329645 s, 31.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:11:51,711::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:11:52,491::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:11:52,576::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0439584 s, 23.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:11:53,088::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:11:53,096::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000794578 s, 5.2 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:11:53,526::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:11:53,527::task::595::TaskManager.Task::(_updateState) Task=`5e9c6c77-4aa4-4f23-9cc3-469fbb6e145a`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:11:53,527::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:11:53,528::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:11:53,528::task::1185::TaskManager.Task::(prepare) Task=`5e9c6c77-4aa4-4f23-9cc3-469fbb6e145a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:11:53,528::task::595::TaskManager.Task::(_updateState) Task=`5e9c6c77-4aa4-4f23-9cc3-469fbb6e145a`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:11:53,528::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:11:53,528::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:11:53,528::task::990::TaskManager.Task::(_decref) Task=`5e9c6c77-4aa4-4f23-9cc3-469fbb6e145a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:11:53,533::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:11:53,533::task::595::TaskManager.Task::(_updateState) Task=`59b9af35-4d41-49fe-a553-154a9b92d7e4`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:11:53,533::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:11:53,534::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`87e6f73b-03ce-4312-b1f0-782bdfcb379a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:11:53,534::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:11:53,534::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:11:53,535::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`87e6f73b-03ce-4312-b1f0-782bdfcb379a`::Granted request >Thread-338::DEBUG::2014-05-13 11:11:53,535::task::827::TaskManager.Task::(resourceAcquired) Task=`59b9af35-4d41-49fe-a553-154a9b92d7e4`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:11:53,535::task::990::TaskManager.Task::(_decref) Task=`59b9af35-4d41-49fe-a553-154a9b92d7e4`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:11:53,536::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:11:53,536::task::1185::TaskManager.Task::(prepare) Task=`59b9af35-4d41-49fe-a553-154a9b92d7e4`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:11:53,536::task::595::TaskManager.Task::(_updateState) Task=`59b9af35-4d41-49fe-a553-154a9b92d7e4`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:11:53,537::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:11:53,537::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:11:53,537::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:11:53,537::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:11:53,537::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:11:53,538::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:11:53,538::task::990::TaskManager.Task::(_decref) Task=`59b9af35-4d41-49fe-a553-154a9b92d7e4`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:11:54,589::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:11:54,669::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0377882 s, 27.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:11:54,731::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:11:56,682::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:11:56,757::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311322 s, 32.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:11:57,750::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:11:58,770::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:11:58,850::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035955 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:12:00,768::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:12:00,863::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:12:00,943::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374562 s, 27.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:12:02,956::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:12:03,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316508 s, 32.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:12:03,104::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:12:03,112::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000706604 s, 5.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:12:03,557::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3256e43f] >Thread-66::DEBUG::2014-05-13 11:12:03,557::task::595::TaskManager.Task::(_updateState) Task=`f52e0e48-b957-43f0-b21d-8f4f4101b89a`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:12:03,558::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:12:03,558::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:12:03,558::task::1185::TaskManager.Task::(prepare) Task=`f52e0e48-b957-43f0-b21d-8f4f4101b89a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:12:03,558::task::595::TaskManager.Task::(_updateState) Task=`f52e0e48-b957-43f0-b21d-8f4f4101b89a`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:12:03,559::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:12:03,559::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:12:03,559::task::990::TaskManager.Task::(_decref) Task=`f52e0e48-b957-43f0-b21d-8f4f4101b89a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:12:03,565::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3256e43f] >Thread-338::DEBUG::2014-05-13 11:12:03,566::task::595::TaskManager.Task::(_updateState) Task=`98c13caf-290d-4b0e-82d4-6d59b6bfb607`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:12:03,566::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:12:03,566::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`df2470fa-17b2-461b-b645-d83cb6208671`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:12:03,566::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:12:03,567::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:12:03,567::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`df2470fa-17b2-461b-b645-d83cb6208671`::Granted request >Thread-338::DEBUG::2014-05-13 11:12:03,567::task::827::TaskManager.Task::(resourceAcquired) Task=`98c13caf-290d-4b0e-82d4-6d59b6bfb607`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:12:03,567::task::990::TaskManager.Task::(_decref) Task=`98c13caf-290d-4b0e-82d4-6d59b6bfb607`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:12:03,568::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:12:03,568::task::1185::TaskManager.Task::(prepare) Task=`98c13caf-290d-4b0e-82d4-6d59b6bfb607`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:12:03,569::task::595::TaskManager.Task::(_updateState) Task=`98c13caf-290d-4b0e-82d4-6d59b6bfb607`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:12:03,569::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:12:03,569::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:12:03,569::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:12:03,570::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:12:03,570::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:12:03,570::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:12:03,570::task::990::TaskManager.Task::(_decref) Task=`98c13caf-290d-4b0e-82d4-6d59b6bfb607`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:12:03,787::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:12:04,144::task::595::TaskManager.Task::(_updateState) Task=`8be98a79-b338-4f65-8416-4921a7152946`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:12:04,169::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:12:04,170::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000706604', 'lastCheck': '1.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:12:04,170::task::1185::TaskManager.Task::(prepare) Task=`8be98a79-b338-4f65-8416-4921a7152946`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000706604', 'lastCheck': '1.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:12:04,170::task::595::TaskManager.Task::(_updateState) Task=`8be98a79-b338-4f65-8416-4921a7152946`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:12:04,170::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:12:04,170::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:12:04,170::task::990::TaskManager.Task::(_decref) Task=`8be98a79-b338-4f65-8416-4921a7152946`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:12:05,045::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:12:05,121::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316439 s, 32.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:12:06,806::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:12:07,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:12:07,540::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.363557 s, 2.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:12:09,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:12:09,631::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0350951 s, 29.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:12:09,827::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:12:11,644::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:12:11,719::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.030567 s, 33.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:12:12,851::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:12:13,119::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:12:13,127::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000634426 s, 6.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:12:13,588::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:12:13,589::task::595::TaskManager.Task::(_updateState) Task=`cb16b02b-99ae-45e9-9ee8-7e59b9f47cf9`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:12:13,589::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:12:13,589::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:12:13,590::task::1185::TaskManager.Task::(prepare) Task=`cb16b02b-99ae-45e9-9ee8-7e59b9f47cf9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:12:13,590::task::595::TaskManager.Task::(_updateState) Task=`cb16b02b-99ae-45e9-9ee8-7e59b9f47cf9`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:12:13,590::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:12:13,590::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:12:13,590::task::990::TaskManager.Task::(_decref) Task=`cb16b02b-99ae-45e9-9ee8-7e59b9f47cf9`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:12:13,595::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:12:13,595::task::595::TaskManager.Task::(_updateState) Task=`2a0f9fa5-e44a-4e9b-bb85-73906c3e94a2`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:12:13,595::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:12:13,596::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9e31e88e-77c0-4984-b2b8-027a7a1c804a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:12:13,596::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:12:13,596::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:12:13,597::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9e31e88e-77c0-4984-b2b8-027a7a1c804a`::Granted request >Thread-338::DEBUG::2014-05-13 11:12:13,597::task::827::TaskManager.Task::(resourceAcquired) Task=`2a0f9fa5-e44a-4e9b-bb85-73906c3e94a2`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:12:13,597::task::990::TaskManager.Task::(_decref) Task=`2a0f9fa5-e44a-4e9b-bb85-73906c3e94a2`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:12:13,598::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:12:13,598::task::1185::TaskManager.Task::(prepare) Task=`2a0f9fa5-e44a-4e9b-bb85-73906c3e94a2`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:12:13,598::task::595::TaskManager.Task::(_updateState) Task=`2a0f9fa5-e44a-4e9b-bb85-73906c3e94a2`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:12:13,599::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:12:13,599::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:12:13,599::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:12:13,599::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:12:13,599::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:12:13,600::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:12:13,600::task::990::TaskManager.Task::(_decref) Task=`2a0f9fa5-e44a-4e9b-bb85-73906c3e94a2`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:12:13,732::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:12:13,810::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0351525 s, 29.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:12:15,823::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:12:15,870::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:12:15,904::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352422 s, 29.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:12:17,917::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:12:18,074::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0300023 s, 34.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:12:18,891::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:12:19,301::task::595::TaskManager.Task::(_updateState) Task=`3cf655fe-0924-41f6-b23e-d3b8c7328f1a`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:12:19,301::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:12:19,302::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000634426', 'lastCheck': '6.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:12:19,302::task::1185::TaskManager.Task::(prepare) Task=`3cf655fe-0924-41f6-b23e-d3b8c7328f1a`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000634426', 'lastCheck': '6.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:12:19,302::task::595::TaskManager.Task::(_updateState) Task=`3cf655fe-0924-41f6-b23e-d3b8c7328f1a`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:12:19,302::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:12:19,302::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:12:19,302::task::990::TaskManager.Task::(_decref) Task=`3cf655fe-0924-41f6-b23e-d3b8c7328f1a`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:12:20,179::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:12:20,541::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.319097 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:12:21,910::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:12:22,554::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:12:22,628::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321132 s, 31.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:12:23,134::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:12:23,143::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000589895 s, 6.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:12:23,618::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5affc0d0] >Thread-66::DEBUG::2014-05-13 11:12:23,618::task::595::TaskManager.Task::(_updateState) Task=`649d1bc4-5078-46ae-8421-ba8d300c49fa`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:12:23,619::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:12:23,619::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:12:23,619::task::1185::TaskManager.Task::(prepare) Task=`649d1bc4-5078-46ae-8421-ba8d300c49fa`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:12:23,619::task::595::TaskManager.Task::(_updateState) Task=`649d1bc4-5078-46ae-8421-ba8d300c49fa`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:12:23,620::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:12:23,620::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:12:23,620::task::990::TaskManager.Task::(_decref) Task=`649d1bc4-5078-46ae-8421-ba8d300c49fa`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:12:23,625::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5affc0d0] >Thread-338::DEBUG::2014-05-13 11:12:23,625::task::595::TaskManager.Task::(_updateState) Task=`df84ee26-d30b-4e5c-bda3-e468f6ce1166`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:12:23,626::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:12:23,626::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4c4aee4f-09f2-4d4a-ab26-0d1ab6e22bc6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:12:23,626::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:12:23,627::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:12:23,627::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4c4aee4f-09f2-4d4a-ab26-0d1ab6e22bc6`::Granted request >Thread-338::DEBUG::2014-05-13 11:12:23,628::task::827::TaskManager.Task::(resourceAcquired) Task=`df84ee26-d30b-4e5c-bda3-e468f6ce1166`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:12:23,628::task::990::TaskManager.Task::(_decref) Task=`df84ee26-d30b-4e5c-bda3-e468f6ce1166`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:12:23,629::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:12:23,629::task::1185::TaskManager.Task::(prepare) Task=`df84ee26-d30b-4e5c-bda3-e468f6ce1166`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:12:23,629::task::595::TaskManager.Task::(_updateState) Task=`df84ee26-d30b-4e5c-bda3-e468f6ce1166`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:12:23,630::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:12:23,630::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:12:23,630::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:12:23,630::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:12:23,630::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:12:23,631::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:12:23,631::task::990::TaskManager.Task::(_decref) Task=`df84ee26-d30b-4e5c-bda3-e468f6ce1166`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:12:24,642::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:12:24,718::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0344064 s, 29.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:12:24,933::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:12:26,731::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:12:26,805::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310006 s, 33.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:12:27,952::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:12:28,819::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:12:28,908::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0454101 s, 22.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:12:30,921::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:12:30,980::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:12:31,002::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362037 s, 28.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:12:33,015::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:12:33,102::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0428515 s, 23.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:12:33,150::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:12:33,158::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000623697 s, 6.6 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:12:33,651::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4dbe430a] >Thread-66::DEBUG::2014-05-13 11:12:33,651::task::595::TaskManager.Task::(_updateState) Task=`6077cc00-77c8-49ce-a4ea-2d852e238e5b`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:12:33,651::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:12:33,652::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:12:33,652::task::1185::TaskManager.Task::(prepare) Task=`6077cc00-77c8-49ce-a4ea-2d852e238e5b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:12:33,652::task::595::TaskManager.Task::(_updateState) Task=`6077cc00-77c8-49ce-a4ea-2d852e238e5b`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:12:33,652::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:12:33,653::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:12:33,653::task::990::TaskManager.Task::(_decref) Task=`6077cc00-77c8-49ce-a4ea-2d852e238e5b`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:12:33,957::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4dbe430a] >Thread-338::DEBUG::2014-05-13 11:12:33,958::task::595::TaskManager.Task::(_updateState) Task=`14cb8d21-f915-42b5-9e57-b692a29e61b1`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:12:33,958::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:12:33,958::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6a156f4e-2eb5-4efa-a685-b45e1892df94`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:12:33,959::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:12:33,959::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:12:33,959::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6a156f4e-2eb5-4efa-a685-b45e1892df94`::Granted request >Thread-338::DEBUG::2014-05-13 11:12:33,959::task::827::TaskManager.Task::(resourceAcquired) Task=`14cb8d21-f915-42b5-9e57-b692a29e61b1`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:12:33,960::task::990::TaskManager.Task::(_decref) Task=`14cb8d21-f915-42b5-9e57-b692a29e61b1`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:12:33,960::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:12:33,960::task::1185::TaskManager.Task::(prepare) Task=`14cb8d21-f915-42b5-9e57-b692a29e61b1`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:12:33,961::task::595::TaskManager.Task::(_updateState) Task=`14cb8d21-f915-42b5-9e57-b692a29e61b1`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:12:33,961::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:12:33,961::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:12:33,961::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:12:33,961::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:12:33,962::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:12:33,962::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:12:33,962::task::990::TaskManager.Task::(_decref) Task=`14cb8d21-f915-42b5-9e57-b692a29e61b1`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:12:33,997::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:12:34,433::task::595::TaskManager.Task::(_updateState) Task=`43efa451-a6d3-4d2e-ba4e-93facd1e9119`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:12:34,433::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:12:34,433::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000623697', 'lastCheck': '1.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:12:34,433::task::1185::TaskManager.Task::(prepare) Task=`43efa451-a6d3-4d2e-ba4e-93facd1e9119`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000623697', 'lastCheck': '1.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:12:34,434::task::595::TaskManager.Task::(_updateState) Task=`43efa451-a6d3-4d2e-ba4e-93facd1e9119`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:12:34,434::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:12:34,434::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:12:34,434::task::990::TaskManager.Task::(_decref) Task=`43efa451-a6d3-4d2e-ba4e-93facd1e9119`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:12:35,115::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:12:35,190::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0327728 s, 31.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:12:37,016::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:12:37,203::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:12:37,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320783 s, 31.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:12:39,291::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:12:39,369::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358251 s, 28.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:12:40,035::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:12:41,382::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:12:41,456::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.03113 s, 32.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:12:43,056::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:12:43,166::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:12:43,174::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00058554 s, 7.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:12:43,469::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:12:43,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036456 s, 28.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:12:43,982::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:12:44,004::task::595::TaskManager.Task::(_updateState) Task=`bdf2a715-9289-4933-80be-ad6b15ead67e`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:12:44,004::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:12:44,004::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:12:44,005::task::1185::TaskManager.Task::(prepare) Task=`bdf2a715-9289-4933-80be-ad6b15ead67e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:12:44,005::task::595::TaskManager.Task::(_updateState) Task=`bdf2a715-9289-4933-80be-ad6b15ead67e`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:12:44,005::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:12:44,005::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:12:44,005::task::990::TaskManager.Task::(_decref) Task=`bdf2a715-9289-4933-80be-ad6b15ead67e`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:12:44,010::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:12:44,010::task::595::TaskManager.Task::(_updateState) Task=`ee65927e-b3d2-4590-9043-036ce53aa6c3`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:12:44,011::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:12:44,011::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`abd69f28-e99a-4f1d-82d3-5d0ff343e9fd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:12:44,011::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:12:44,012::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:12:44,012::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`abd69f28-e99a-4f1d-82d3-5d0ff343e9fd`::Granted request >Thread-338::DEBUG::2014-05-13 11:12:44,012::task::827::TaskManager.Task::(resourceAcquired) Task=`ee65927e-b3d2-4590-9043-036ce53aa6c3`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:12:44,012::task::990::TaskManager.Task::(_decref) Task=`ee65927e-b3d2-4590-9043-036ce53aa6c3`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:12:44,013::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:12:44,013::task::1185::TaskManager.Task::(prepare) Task=`ee65927e-b3d2-4590-9043-036ce53aa6c3`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:12:44,014::task::595::TaskManager.Task::(_updateState) Task=`ee65927e-b3d2-4590-9043-036ce53aa6c3`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:12:44,014::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:12:44,014::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:12:44,014::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:12:44,014::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:12:44,015::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:12:44,015::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:12:44,015::task::990::TaskManager.Task::(_decref) Task=`ee65927e-b3d2-4590-9043-036ce53aa6c3`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:12:45,565::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:12:45,642::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0349621 s, 29.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:12:46,074::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:12:47,655::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:12:48,027::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.32762 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:12:49,094::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:12:49,574::task::595::TaskManager.Task::(_updateState) Task=`bce57a2d-0745-4724-bcc8-2766a41990c6`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:12:49,574::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:12:49,574::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00058554', 'lastCheck': '6.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:12:49,574::task::1185::TaskManager.Task::(prepare) Task=`bce57a2d-0745-4724-bcc8-2766a41990c6`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00058554', 'lastCheck': '6.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:12:49,574::task::595::TaskManager.Task::(_updateState) Task=`bce57a2d-0745-4724-bcc8-2766a41990c6`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:12:49,575::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:12:49,575::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:12:49,575::task::990::TaskManager.Task::(_decref) Task=`bce57a2d-0745-4724-bcc8-2766a41990c6`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:12:50,040::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:12:50,114::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0304942 s, 33.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:12:52,114::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:12:52,127::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:12:52,199::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0304898 s, 33.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:12:53,182::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:12:53,190::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000629596 s, 6.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:12:54,034::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [12ac301a] >Thread-66::DEBUG::2014-05-13 11:12:54,034::task::595::TaskManager.Task::(_updateState) Task=`e7b25533-22b2-45a4-8531-4fdded561841`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:12:54,035::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:12:54,035::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:12:54,035::task::1185::TaskManager.Task::(prepare) Task=`e7b25533-22b2-45a4-8531-4fdded561841`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:12:54,035::task::595::TaskManager.Task::(_updateState) Task=`e7b25533-22b2-45a4-8531-4fdded561841`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:12:54,036::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:12:54,036::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:12:54,036::task::990::TaskManager.Task::(_decref) Task=`e7b25533-22b2-45a4-8531-4fdded561841`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:12:54,041::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [12ac301a] >Thread-338::DEBUG::2014-05-13 11:12:54,041::task::595::TaskManager.Task::(_updateState) Task=`ddb255fd-c914-474e-9d21-4ba35acbae54`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:12:54,042::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:12:54,042::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`883d3a21-cde8-40a1-be4a-9caf5655c9d1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:12:54,042::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:12:54,043::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:12:54,043::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`883d3a21-cde8-40a1-be4a-9caf5655c9d1`::Granted request >Thread-338::DEBUG::2014-05-13 11:12:54,043::task::827::TaskManager.Task::(resourceAcquired) Task=`ddb255fd-c914-474e-9d21-4ba35acbae54`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:12:54,043::task::990::TaskManager.Task::(_decref) Task=`ddb255fd-c914-474e-9d21-4ba35acbae54`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:12:54,044::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:12:54,044::task::1185::TaskManager.Task::(prepare) Task=`ddb255fd-c914-474e-9d21-4ba35acbae54`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:12:54,045::task::595::TaskManager.Task::(_updateState) Task=`ddb255fd-c914-474e-9d21-4ba35acbae54`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:12:54,045::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:12:54,045::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:12:54,045::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:12:54,046::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:12:54,046::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:12:54,046::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:12:54,046::task::990::TaskManager.Task::(_decref) Task=`ddb255fd-c914-474e-9d21-4ba35acbae54`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:12:54,213::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:12:54,296::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0368099 s, 27.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:12:55,147::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:12:56,309::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:12:56,383::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307542 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:12:58,169::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:12:58,396::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:12:58,477::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353527 s, 29.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:13:00,490::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:00,582::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0479978 s, 21.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:13:01,195::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:13:02,595::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:02,670::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0329344 s, 31.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:13:03,204::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:13:03,212::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00087207 s, 4.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:13:04,065::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:13:04,066::task::595::TaskManager.Task::(_updateState) Task=`55e6d926-c2d9-4220-8142-130161b74ddd`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:13:04,066::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:13:04,066::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:13:04,067::task::1185::TaskManager.Task::(prepare) Task=`55e6d926-c2d9-4220-8142-130161b74ddd`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:13:04,067::task::595::TaskManager.Task::(_updateState) Task=`55e6d926-c2d9-4220-8142-130161b74ddd`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:13:04,067::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:13:04,067::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:13:04,067::task::990::TaskManager.Task::(_decref) Task=`55e6d926-c2d9-4220-8142-130161b74ddd`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:13:04,074::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:13:04,074::task::595::TaskManager.Task::(_updateState) Task=`a9b32f2d-9063-44fc-a02f-8f841a359254`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:13:04,074::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:13:04,075::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ab87ef74-e220-4fcb-ae94-4eb0655e0d2d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:13:04,075::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:13:04,075::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:13:04,076::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ab87ef74-e220-4fcb-ae94-4eb0655e0d2d`::Granted request >Thread-338::DEBUG::2014-05-13 11:13:04,076::task::827::TaskManager.Task::(resourceAcquired) Task=`a9b32f2d-9063-44fc-a02f-8f841a359254`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:13:04,076::task::990::TaskManager.Task::(_decref) Task=`a9b32f2d-9063-44fc-a02f-8f841a359254`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:13:04,077::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:13:04,077::task::1185::TaskManager.Task::(prepare) Task=`a9b32f2d-9063-44fc-a02f-8f841a359254`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:13:04,077::task::595::TaskManager.Task::(_updateState) Task=`a9b32f2d-9063-44fc-a02f-8f841a359254`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:13:04,077::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:13:04,078::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:13:04,078::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:13:04,078::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:13:04,078::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:13:04,078::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:13:04,079::task::990::TaskManager.Task::(_decref) Task=`a9b32f2d-9063-44fc-a02f-8f841a359254`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:13:04,213::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:13:04,683::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 11:13:04,706::task::595::TaskManager.Task::(_updateState) Task=`c82aa91e-d73d-4d00-a4db-e4fefa2722c3`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:13:04,707::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:13:04,707::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00087207', 'lastCheck': '1.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:13:04,707::task::1185::TaskManager.Task::(prepare) Task=`c82aa91e-d73d-4d00-a4db-e4fefa2722c3`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00087207', 'lastCheck': '1.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:13:04,708::task::595::TaskManager.Task::(_updateState) Task=`c82aa91e-d73d-4d00-a4db-e4fefa2722c3`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:13:04,708::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:13:04,708::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:13:04,708::task::990::TaskManager.Task::(_decref) Task=`c82aa91e-d73d-4d00-a4db-e4fefa2722c3`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:13:04,766::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0380079 s, 26.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:13:06,779::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:06,859::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037815 s, 27.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:13:07,231::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:13:08,872::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:08,952::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0376662 s, 27.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:13:10,252::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:13:10,965::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:11,039::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303422 s, 33.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:13:13,052::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:13,131::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0346645 s, 29.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:13:13,219::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:13:13,228::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000613183 s, 6.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:13:13,272::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:13:14,097::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:13:14,097::task::595::TaskManager.Task::(_updateState) Task=`401b8160-90e2-4a06-91ce-f4412ccc280a`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:13:14,098::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:13:14,098::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:13:14,098::task::1185::TaskManager.Task::(prepare) Task=`401b8160-90e2-4a06-91ce-f4412ccc280a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:13:14,098::task::595::TaskManager.Task::(_updateState) Task=`401b8160-90e2-4a06-91ce-f4412ccc280a`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:13:14,099::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:13:14,099::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:13:14,099::task::990::TaskManager.Task::(_decref) Task=`401b8160-90e2-4a06-91ce-f4412ccc280a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:13:14,104::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:13:14,105::task::595::TaskManager.Task::(_updateState) Task=`2e70ea0b-5e90-4696-bef5-ae7285482949`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:13:14,105::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:13:14,105::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6e21f21d-5a3c-49f4-b041-b0a8dbe52914`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:13:14,106::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:13:14,106::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:13:14,106::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6e21f21d-5a3c-49f4-b041-b0a8dbe52914`::Granted request >Thread-338::DEBUG::2014-05-13 11:13:14,106::task::827::TaskManager.Task::(resourceAcquired) Task=`2e70ea0b-5e90-4696-bef5-ae7285482949`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:13:14,107::task::990::TaskManager.Task::(_decref) Task=`2e70ea0b-5e90-4696-bef5-ae7285482949`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:13:14,107::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:13:14,108::task::1185::TaskManager.Task::(prepare) Task=`2e70ea0b-5e90-4696-bef5-ae7285482949`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:13:14,108::task::595::TaskManager.Task::(_updateState) Task=`2e70ea0b-5e90-4696-bef5-ae7285482949`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:13:14,108::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:13:14,108::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:13:14,108::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:13:14,109::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:13:14,109::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:13:14,109::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:13:14,109::task::990::TaskManager.Task::(_decref) Task=`2e70ea0b-5e90-4696-bef5-ae7285482949`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:13:15,143::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:15,512::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.317259 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:13:16,293::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:13:17,524::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:17,597::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.029532 s, 34.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:13:19,312::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:13:19,610::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:19,685::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316706 s, 32.3 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:13:19,851::task::595::TaskManager.Task::(_updateState) Task=`4749bf64-0ffa-479b-b58e-7dad4416abb1`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:13:19,851::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:13:19,851::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000613183', 'lastCheck': '6.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:13:19,852::task::1185::TaskManager.Task::(prepare) Task=`4749bf64-0ffa-479b-b58e-7dad4416abb1`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000613183', 'lastCheck': '6.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:13:19,852::task::595::TaskManager.Task::(_updateState) Task=`4749bf64-0ffa-479b-b58e-7dad4416abb1`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:13:19,852::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:13:19,852::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:13:19,852::task::990::TaskManager.Task::(_decref) Task=`4749bf64-0ffa-479b-b58e-7dad4416abb1`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:13:21,698::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:21,776::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0349335 s, 29.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:13:22,331::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:13:23,234::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:13:23,243::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00064871 s, 6.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:13:23,790::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:23,867::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0346673 s, 29.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:13:24,134::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:13:24,134::task::595::TaskManager.Task::(_updateState) Task=`ef4ff2bd-531a-49a0-b8db-0f81a0da8bd4`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:13:24,135::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:13:24,135::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:13:24,135::task::1185::TaskManager.Task::(prepare) Task=`ef4ff2bd-531a-49a0-b8db-0f81a0da8bd4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:13:24,135::task::595::TaskManager.Task::(_updateState) Task=`ef4ff2bd-531a-49a0-b8db-0f81a0da8bd4`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:13:24,135::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:13:24,136::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:13:24,136::task::990::TaskManager.Task::(_decref) Task=`ef4ff2bd-531a-49a0-b8db-0f81a0da8bd4`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:13:24,140::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:13:24,140::task::595::TaskManager.Task::(_updateState) Task=`0a11561c-99c5-4fc3-9acb-cd9656b582e6`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:13:24,141::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:13:24,141::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1148ae2f-0ef3-4c1c-b605-815b77c7dffb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:13:24,141::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:13:24,142::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:13:24,142::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1148ae2f-0ef3-4c1c-b605-815b77c7dffb`::Granted request >Thread-338::DEBUG::2014-05-13 11:13:24,142::task::827::TaskManager.Task::(resourceAcquired) Task=`0a11561c-99c5-4fc3-9acb-cd9656b582e6`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:13:24,142::task::990::TaskManager.Task::(_decref) Task=`0a11561c-99c5-4fc3-9acb-cd9656b582e6`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:13:24,143::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:13:24,143::task::1185::TaskManager.Task::(prepare) Task=`0a11561c-99c5-4fc3-9acb-cd9656b582e6`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:13:24,143::task::595::TaskManager.Task::(_updateState) Task=`0a11561c-99c5-4fc3-9acb-cd9656b582e6`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:13:24,144::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:13:24,144::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:13:24,144::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:13:24,144::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:13:24,144::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:13:24,145::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:13:24,145::task::990::TaskManager.Task::(_decref) Task=`0a11561c-99c5-4fc3-9acb-cd9656b582e6`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:13:25,351::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:13:25,881::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:25,960::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312577 s, 32.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:13:27,972::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:28,057::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0416551 s, 24.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:13:28,372::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:13:30,070::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:30,160::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0469698 s, 21.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:13:31,394::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:13:32,173::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:32,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303041 s, 33.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:13:33,250::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:13:33,274::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.0155407 s, 264 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:13:34,161::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:13:34,161::task::595::TaskManager.Task::(_updateState) Task=`33fef334-5d80-4b81-a7bb-cc25cc730136`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:13:34,162::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:13:34,162::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:13:34,162::task::1185::TaskManager.Task::(prepare) Task=`33fef334-5d80-4b81-a7bb-cc25cc730136`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:13:34,162::task::595::TaskManager.Task::(_updateState) Task=`33fef334-5d80-4b81-a7bb-cc25cc730136`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:13:34,163::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:13:34,163::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:13:34,163::task::990::TaskManager.Task::(_decref) Task=`33fef334-5d80-4b81-a7bb-cc25cc730136`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:13:34,167::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:13:34,167::task::595::TaskManager.Task::(_updateState) Task=`dd007350-6fba-4dce-9ea7-91968f090f89`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:13:34,168::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:13:34,168::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e0dd08c2-0d83-4348-b2b1-a040ec23ddf9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:13:34,168::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:13:34,168::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:13:34,169::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e0dd08c2-0d83-4348-b2b1-a040ec23ddf9`::Granted request >Thread-338::DEBUG::2014-05-13 11:13:34,170::task::827::TaskManager.Task::(resourceAcquired) Task=`dd007350-6fba-4dce-9ea7-91968f090f89`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:13:34,170::task::990::TaskManager.Task::(_decref) Task=`dd007350-6fba-4dce-9ea7-91968f090f89`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:13:34,171::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:13:34,171::task::1185::TaskManager.Task::(prepare) Task=`dd007350-6fba-4dce-9ea7-91968f090f89`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:13:34,171::task::595::TaskManager.Task::(_updateState) Task=`dd007350-6fba-4dce-9ea7-91968f090f89`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:13:34,171::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:13:34,171::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:13:34,172::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:13:34,172::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:13:34,172::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:13:34,172::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:13:34,172::task::990::TaskManager.Task::(_decref) Task=`dd007350-6fba-4dce-9ea7-91968f090f89`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:13:34,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:34,353::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319126 s, 32.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:13:34,411::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:13:34,987::task::595::TaskManager.Task::(_updateState) Task=`1de5022c-4350-4097-ab9a-63236b3fc6a7`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:13:34,987::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:13:34,987::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0155407', 'lastCheck': '1.7', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:13:34,987::task::1185::TaskManager.Task::(prepare) Task=`1de5022c-4350-4097-ab9a-63236b3fc6a7`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0155407', 'lastCheck': '1.7', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:13:34,987::task::595::TaskManager.Task::(_updateState) Task=`1de5022c-4350-4097-ab9a-63236b3fc6a7`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:13:34,988::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:13:34,988::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:13:34,988::task::990::TaskManager.Task::(_decref) Task=`1de5022c-4350-4097-ab9a-63236b3fc6a7`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:13:36,366::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:36,446::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353461 s, 29.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:13:37,430::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:13:38,459::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:38,534::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.032347 s, 31.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:13:40,547::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:13:40,614::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:13:40,625::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0328364 s, 31.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:13:42,637::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:42,728::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0476166 s, 21.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:13:43,281::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:13:43,289::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000724272 s, 5.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:13:43,634::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:13:44,191::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:13:44,191::task::595::TaskManager.Task::(_updateState) Task=`caf63c28-5dbe-4d60-b0c5-3b0108c59cff`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:13:44,191::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:13:44,192::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:13:44,192::task::1185::TaskManager.Task::(prepare) Task=`caf63c28-5dbe-4d60-b0c5-3b0108c59cff`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:13:44,192::task::595::TaskManager.Task::(_updateState) Task=`caf63c28-5dbe-4d60-b0c5-3b0108c59cff`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:13:44,192::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:13:44,192::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:13:44,193::task::990::TaskManager.Task::(_decref) Task=`caf63c28-5dbe-4d60-b0c5-3b0108c59cff`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:13:44,197::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:13:44,197::task::595::TaskManager.Task::(_updateState) Task=`dda1fe17-7327-4b65-be67-f87092094936`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:13:44,197::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:13:44,198::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9b2fe52c-0803-4fb3-8a76-ed1cad0c40fb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:13:44,198::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:13:44,198::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:13:44,199::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9b2fe52c-0803-4fb3-8a76-ed1cad0c40fb`::Granted request >Thread-338::DEBUG::2014-05-13 11:13:44,199::task::827::TaskManager.Task::(resourceAcquired) Task=`dda1fe17-7327-4b65-be67-f87092094936`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:13:44,199::task::990::TaskManager.Task::(_decref) Task=`dda1fe17-7327-4b65-be67-f87092094936`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:13:44,200::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:13:44,200::task::1185::TaskManager.Task::(prepare) Task=`dda1fe17-7327-4b65-be67-f87092094936`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:13:44,200::task::595::TaskManager.Task::(_updateState) Task=`dda1fe17-7327-4b65-be67-f87092094936`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:13:44,200::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:13:44,201::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:13:44,201::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:13:44,201::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:13:44,201::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:13:44,201::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:13:44,202::task::990::TaskManager.Task::(_decref) Task=`dda1fe17-7327-4b65-be67-f87092094936`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:13:44,741::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:44,828::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.043621 s, 23.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:13:46,662::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:13:46,841::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:46,927::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0434433 s, 23.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:13:48,940::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:49,026::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0431946 s, 23.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:13:49,683::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:13:50,121::task::595::TaskManager.Task::(_updateState) Task=`8a1b808d-9776-42e7-a49b-896c2832e5dd`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:13:50,121::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:13:50,121::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000724272', 'lastCheck': '6.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:13:50,122::task::1185::TaskManager.Task::(prepare) Task=`8a1b808d-9776-42e7-a49b-896c2832e5dd`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000724272', 'lastCheck': '6.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:13:50,122::task::595::TaskManager.Task::(_updateState) Task=`8a1b808d-9776-42e7-a49b-896c2832e5dd`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:13:50,122::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:13:50,122::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:13:50,122::task::990::TaskManager.Task::(_decref) Task=`8a1b808d-9776-42e7-a49b-896c2832e5dd`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:13:51,039::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:51,113::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.02995 s, 34.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:13:52,709::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:13:53,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:53,200::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307181 s, 33.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:13:53,296::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:13:53,305::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000619971 s, 6.6 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:13:54,219::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:13:54,222::task::595::TaskManager.Task::(_updateState) Task=`9c3217a1-a496-4522-a6eb-67fb4ec09c91`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:13:54,222::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:13:54,223::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:13:54,223::task::1185::TaskManager.Task::(prepare) Task=`9c3217a1-a496-4522-a6eb-67fb4ec09c91`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:13:54,223::task::595::TaskManager.Task::(_updateState) Task=`9c3217a1-a496-4522-a6eb-67fb4ec09c91`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:13:54,223::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:13:54,223::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:13:54,224::task::990::TaskManager.Task::(_decref) Task=`9c3217a1-a496-4522-a6eb-67fb4ec09c91`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:13:54,229::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:13:54,229::task::595::TaskManager.Task::(_updateState) Task=`69a57297-a844-440c-acef-dff6cdbda9e7`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:13:54,230::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:13:54,230::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`da5d4086-07c6-4308-9d34-8da38bfdcacf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:13:54,230::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:13:54,231::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:13:54,231::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`da5d4086-07c6-4308-9d34-8da38bfdcacf`::Granted request >Thread-338::DEBUG::2014-05-13 11:13:54,231::task::827::TaskManager.Task::(resourceAcquired) Task=`69a57297-a844-440c-acef-dff6cdbda9e7`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:13:54,231::task::990::TaskManager.Task::(_decref) Task=`69a57297-a844-440c-acef-dff6cdbda9e7`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:13:54,232::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:13:54,232::task::1185::TaskManager.Task::(prepare) Task=`69a57297-a844-440c-acef-dff6cdbda9e7`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:13:54,233::task::595::TaskManager.Task::(_updateState) Task=`69a57297-a844-440c-acef-dff6cdbda9e7`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:13:54,233::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:13:54,233::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:13:54,233::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:13:54,233::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:13:54,234::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:13:54,234::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:13:54,234::task::990::TaskManager.Task::(_decref) Task=`69a57297-a844-440c-acef-dff6cdbda9e7`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:13:55,217::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:55,631::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.37163 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:13:55,728::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:13:57,644::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:57,724::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036367 s, 28.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:13:58,747::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:13:59,737::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:13:59,816::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357581 s, 28.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:14:01,765::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:14:01,829::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:01,903::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315952 s, 32.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:14:03,312::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:14:03,321::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.0015914 s, 2.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:14:03,917::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:03,996::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0366638 s, 27.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:14:04,257::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:14:04,257::task::595::TaskManager.Task::(_updateState) Task=`e2704b13-eb60-4e6c-9b0c-779953faf6b9`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:14:04,257::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:14:04,258::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:14:04,258::task::1185::TaskManager.Task::(prepare) Task=`e2704b13-eb60-4e6c-9b0c-779953faf6b9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:14:04,258::task::595::TaskManager.Task::(_updateState) Task=`e2704b13-eb60-4e6c-9b0c-779953faf6b9`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:14:04,258::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:14:04,258::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:14:04,258::task::990::TaskManager.Task::(_decref) Task=`e2704b13-eb60-4e6c-9b0c-779953faf6b9`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:14:04,263::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:14:04,263::task::595::TaskManager.Task::(_updateState) Task=`a420f77e-f86f-4ee4-b1b6-335a46047a03`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:14:04,263::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:14:04,264::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4b191716-c847-4f00-aa2f-cb4b4d4f5a88`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:14:04,264::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:14:04,264::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:14:04,264::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4b191716-c847-4f00-aa2f-cb4b4d4f5a88`::Granted request >Thread-338::DEBUG::2014-05-13 11:14:04,265::task::827::TaskManager.Task::(resourceAcquired) Task=`a420f77e-f86f-4ee4-b1b6-335a46047a03`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:14:04,265::task::990::TaskManager.Task::(_decref) Task=`a420f77e-f86f-4ee4-b1b6-335a46047a03`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:14:04,265::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:14:04,266::task::1185::TaskManager.Task::(prepare) Task=`a420f77e-f86f-4ee4-b1b6-335a46047a03`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:14:04,266::task::595::TaskManager.Task::(_updateState) Task=`a420f77e-f86f-4ee4-b1b6-335a46047a03`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:14:04,266::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:14:04,266::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:14:04,266::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:14:04,266::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:14:04,267::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:14:04,267::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:14:04,267::task::990::TaskManager.Task::(_decref) Task=`a420f77e-f86f-4ee4-b1b6-335a46047a03`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:14:04,783::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:14:05,255::task::595::TaskManager.Task::(_updateState) Task=`046c9093-0bd1-4e38-a741-55b3abba2655`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:14:05,256::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:14:05,256::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0015914', 'lastCheck': '1.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:14:05,256::task::1185::TaskManager.Task::(prepare) Task=`046c9093-0bd1-4e38-a741-55b3abba2655`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0015914', 'lastCheck': '1.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:14:05,256::task::595::TaskManager.Task::(_updateState) Task=`046c9093-0bd1-4e38-a741-55b3abba2655`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:14:05,256::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:14:05,256::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:14:05,256::task::990::TaskManager.Task::(_decref) Task=`046c9093-0bd1-4e38-a741-55b3abba2655`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:14:06,009::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:06,083::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305514 s, 33.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:14:07,801::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:14:08,096::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:08,174::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0344388 s, 29.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:14:10,187::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:10,260::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0301569 s, 34.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:14:10,841::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:14:12,273::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:12,380::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0499528 s, 20.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:14:13,328::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:14:13,341::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00599594 s, 683 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:14:13,866::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:14:14,285::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:14:14,286::task::595::TaskManager.Task::(_updateState) Task=`be865759-8dd4-481a-a4f3-2bd9933131b7`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:14:14,286::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:14:14,286::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:14:14,286::task::1185::TaskManager.Task::(prepare) Task=`be865759-8dd4-481a-a4f3-2bd9933131b7`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:14:14,287::task::595::TaskManager.Task::(_updateState) Task=`be865759-8dd4-481a-a4f3-2bd9933131b7`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:14:14,287::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:14:14,287::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:14:14,287::task::990::TaskManager.Task::(_decref) Task=`be865759-8dd4-481a-a4f3-2bd9933131b7`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:14:14,291::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:14:14,292::task::595::TaskManager.Task::(_updateState) Task=`ef0d9344-b1da-4a94-b016-9457cb8f5fed`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:14:14,292::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:14:14,292::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4f4b3889-f899-42e7-8d88-faed2effdd7a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:14:14,293::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:14:14,293::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:14:14,293::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4f4b3889-f899-42e7-8d88-faed2effdd7a`::Granted request >Thread-338::DEBUG::2014-05-13 11:14:14,294::task::827::TaskManager.Task::(resourceAcquired) Task=`ef0d9344-b1da-4a94-b016-9457cb8f5fed`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:14:14,294::task::990::TaskManager.Task::(_decref) Task=`ef0d9344-b1da-4a94-b016-9457cb8f5fed`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:14:14,295::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:14:14,295::task::1185::TaskManager.Task::(prepare) Task=`ef0d9344-b1da-4a94-b016-9457cb8f5fed`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:14:14,295::task::595::TaskManager.Task::(_updateState) Task=`ef0d9344-b1da-4a94-b016-9457cb8f5fed`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:14:14,295::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:14:14,295::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:14:14,296::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:14:14,297::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:14:14,297::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:14:14,297::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:14:14,297::task::990::TaskManager.Task::(_decref) Task=`ef0d9344-b1da-4a94-b016-9457cb8f5fed`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:14:14,393::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:14,475::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0375694 s, 27.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:14:16,488::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:16,563::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317509 s, 32.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:14:16,884::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:14:18,576::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:18,656::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0366315 s, 28.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:14:19,909::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:14:20,394::task::595::TaskManager.Task::(_updateState) Task=`8fff62f1-9023-4d3e-9178-6c078432d593`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:14:20,394::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:14:20,394::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00599594', 'lastCheck': '7.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:14:20,394::task::1185::TaskManager.Task::(prepare) Task=`8fff62f1-9023-4d3e-9178-6c078432d593`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00599594', 'lastCheck': '7.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:14:20,394::task::595::TaskManager.Task::(_updateState) Task=`8fff62f1-9023-4d3e-9178-6c078432d593`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:14:20,395::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:14:20,395::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:14:20,395::task::990::TaskManager.Task::(_decref) Task=`8fff62f1-9023-4d3e-9178-6c078432d593`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:14:20,669::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:20,748::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360237 s, 28.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:14:22,761::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:22,836::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0327474 s, 31.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:14:22,930::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:14:23,348::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:14:23,357::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000650825 s, 6.3 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:14:24,314::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:14:24,314::task::595::TaskManager.Task::(_updateState) Task=`1145fee5-f4ab-4cff-ac99-337d58859a16`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:14:24,315::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:14:24,315::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:14:24,315::task::1185::TaskManager.Task::(prepare) Task=`1145fee5-f4ab-4cff-ac99-337d58859a16`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:14:24,315::task::595::TaskManager.Task::(_updateState) Task=`1145fee5-f4ab-4cff-ac99-337d58859a16`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:14:24,316::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:14:24,316::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:14:24,316::task::990::TaskManager.Task::(_decref) Task=`1145fee5-f4ab-4cff-ac99-337d58859a16`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:14:24,320::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:14:24,321::task::595::TaskManager.Task::(_updateState) Task=`628de597-fefc-4c93-a102-f9262a231fb8`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:14:24,321::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:14:24,321::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9851bb9c-f7df-43d5-a7b0-e9731f47bf72`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:14:24,322::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:14:24,322::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:14:24,322::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9851bb9c-f7df-43d5-a7b0-e9731f47bf72`::Granted request >Thread-338::DEBUG::2014-05-13 11:14:24,322::task::827::TaskManager.Task::(resourceAcquired) Task=`628de597-fefc-4c93-a102-f9262a231fb8`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:14:24,323::task::990::TaskManager.Task::(_decref) Task=`628de597-fefc-4c93-a102-f9262a231fb8`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:14:24,323::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:14:24,324::task::1185::TaskManager.Task::(prepare) Task=`628de597-fefc-4c93-a102-f9262a231fb8`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:14:24,324::task::595::TaskManager.Task::(_updateState) Task=`628de597-fefc-4c93-a102-f9262a231fb8`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:14:24,324::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:14:24,324::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:14:24,324::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:14:24,325::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:14:24,325::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:14:24,325::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:14:24,325::task::990::TaskManager.Task::(_decref) Task=`628de597-fefc-4c93-a102-f9262a231fb8`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:14:24,850::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:24,942::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0485218 s, 21.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:14:25,950::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:14:26,955::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:27,030::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0329421 s, 31.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:14:28,968::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:14:29,043::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:29,122::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357445 s, 28.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:14:31,135::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:31,215::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0380033 s, 26.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:14:31,986::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:14:33,225::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:33,300::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303742 s, 33.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:14:33,364::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:14:33,373::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00171449 s, 2.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:14:34,343::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:14:34,366::task::595::TaskManager.Task::(_updateState) Task=`640ca2e6-0b2d-4b61-84ba-b5dcf56ae67f`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:14:34,366::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:14:34,366::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:14:34,367::task::1185::TaskManager.Task::(prepare) Task=`640ca2e6-0b2d-4b61-84ba-b5dcf56ae67f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:14:34,367::task::595::TaskManager.Task::(_updateState) Task=`640ca2e6-0b2d-4b61-84ba-b5dcf56ae67f`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:14:34,367::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:14:34,367::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:14:34,367::task::990::TaskManager.Task::(_decref) Task=`640ca2e6-0b2d-4b61-84ba-b5dcf56ae67f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:14:34,373::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:14:34,373::task::595::TaskManager.Task::(_updateState) Task=`d00acbb5-c752-48b1-a851-eaf76db1a51e`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:14:34,374::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:14:34,374::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fa666643-57ea-4ff3-822b-0584b1b656fe`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:14:34,374::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:14:34,374::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:14:34,375::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fa666643-57ea-4ff3-822b-0584b1b656fe`::Granted request >Thread-338::DEBUG::2014-05-13 11:14:34,375::task::827::TaskManager.Task::(resourceAcquired) Task=`d00acbb5-c752-48b1-a851-eaf76db1a51e`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:14:34,375::task::990::TaskManager.Task::(_decref) Task=`d00acbb5-c752-48b1-a851-eaf76db1a51e`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:14:34,376::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:14:34,376::task::1185::TaskManager.Task::(prepare) Task=`d00acbb5-c752-48b1-a851-eaf76db1a51e`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:14:34,376::task::595::TaskManager.Task::(_updateState) Task=`d00acbb5-c752-48b1-a851-eaf76db1a51e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:14:34,376::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:14:34,377::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:14:34,377::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:14:34,377::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:14:34,377::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:14:34,377::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:14:34,377::task::990::TaskManager.Task::(_decref) Task=`d00acbb5-c752-48b1-a851-eaf76db1a51e`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:14:35,006::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:14:35,313::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:35,389::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312936 s, 32.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:14:35,525::task::595::TaskManager.Task::(_updateState) Task=`a3a36218-05cb-4447-8312-b5e1fb542993`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:14:35,525::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:14:35,525::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00171449', 'lastCheck': '2.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:14:35,526::task::1185::TaskManager.Task::(prepare) Task=`a3a36218-05cb-4447-8312-b5e1fb542993`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00171449', 'lastCheck': '2.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:14:35,526::task::595::TaskManager.Task::(_updateState) Task=`a3a36218-05cb-4447-8312-b5e1fb542993`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:14:35,526::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:14:35,526::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:14:35,526::task::990::TaskManager.Task::(_decref) Task=`a3a36218-05cb-4447-8312-b5e1fb542993`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:14:37,402::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:37,770::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.325429 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:14:38,024::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:14:39,783::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:39,860::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0345877 s, 29.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:14:41,042::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:14:41,873::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:41,949::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0334972 s, 30.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:14:43,379::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:14:43,388::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000549368 s, 7.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:14:43,962::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:44,043::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0378285 s, 27.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:14:44,060::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:14:44,397::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:14:44,397::task::595::TaskManager.Task::(_updateState) Task=`5e36f422-2abf-440b-9031-5f0e497a1377`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:14:44,397::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:14:44,398::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:14:44,398::task::1185::TaskManager.Task::(prepare) Task=`5e36f422-2abf-440b-9031-5f0e497a1377`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:14:44,398::task::595::TaskManager.Task::(_updateState) Task=`5e36f422-2abf-440b-9031-5f0e497a1377`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:14:44,398::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:14:44,398::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:14:44,399::task::990::TaskManager.Task::(_decref) Task=`5e36f422-2abf-440b-9031-5f0e497a1377`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:14:44,403::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:14:44,404::task::595::TaskManager.Task::(_updateState) Task=`a6523d5d-782d-4413-9781-72bf0d96a359`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:14:44,404::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:14:44,404::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5a414e0b-5f25-4df8-8c02-3bcc0cf2f48e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:14:44,405::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:14:44,405::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:14:44,405::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5a414e0b-5f25-4df8-8c02-3bcc0cf2f48e`::Granted request >Thread-338::DEBUG::2014-05-13 11:14:44,406::task::827::TaskManager.Task::(resourceAcquired) Task=`a6523d5d-782d-4413-9781-72bf0d96a359`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:14:44,407::task::990::TaskManager.Task::(_decref) Task=`a6523d5d-782d-4413-9781-72bf0d96a359`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:14:44,407::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:14:44,408::task::1185::TaskManager.Task::(prepare) Task=`a6523d5d-782d-4413-9781-72bf0d96a359`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:14:44,408::task::595::TaskManager.Task::(_updateState) Task=`a6523d5d-782d-4413-9781-72bf0d96a359`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:14:44,408::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:14:44,408::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:14:44,408::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:14:44,409::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:14:44,409::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:14:44,409::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:14:44,409::task::990::TaskManager.Task::(_decref) Task=`a6523d5d-782d-4413-9781-72bf0d96a359`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:14:46,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:46,138::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0378615 s, 27.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:14:47,079::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:14:48,151::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:48,227::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318152 s, 32.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:14:50,100::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:14:50,240::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:50,314::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310811 s, 32.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:14:50,659::task::595::TaskManager.Task::(_updateState) Task=`e17d91db-abae-4cf0-b4a1-43c29d1ebe30`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:14:50,660::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:14:50,660::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000549368', 'lastCheck': '7.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:14:50,660::task::1185::TaskManager.Task::(prepare) Task=`e17d91db-abae-4cf0-b4a1-43c29d1ebe30`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000549368', 'lastCheck': '7.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:14:50,660::task::595::TaskManager.Task::(_updateState) Task=`e17d91db-abae-4cf0-b4a1-43c29d1ebe30`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:14:50,661::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:14:50,661::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:14:50,661::task::990::TaskManager.Task::(_decref) Task=`e17d91db-abae-4cf0-b4a1-43c29d1ebe30`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:14:52,327::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:52,412::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0414673 s, 24.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:14:53,118::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:14:53,392::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 41830dad-f012-4b4d-b636-863f66d5ca4b >Thread-322::DEBUG::2014-05-13 11:14:53,393::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-322::DEBUG::2014-05-13 11:14:53,395::blockSD::336::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:14:53,404::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000780557 s, 2.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:14:53,405::misc::287::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000780557 s, 2.6 MB/s'], size: 2048 >Thread-322::DEBUG::2014-05-13 11:14:53,405::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-322::WARNING::2014-05-13 11:14:53,405::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-322::DEBUG::2014-05-13 11:14:53,405::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-322::DEBUG::2014-05-13 11:14:53,406::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:14:53,406::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:14:53,406::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:14:53,407::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:14:53,407::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:14:53,408::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 11:14:53,691::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:14:53,692::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:14:53,693::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'LOCKPOLICY=', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', 'DESCRIPTION=ISCSI', 'LOGBLKSIZE=512', 'TYPE=ISCSI', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'PHYBLKSIZE=512', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', 'CLASS=Data', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'POOL_DESCRIPTION=DC_2', 'POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'MASTER_VERSION=1', 'ROLE=Master', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', '_SHA_CKSUM=28bb2d7cc60e0d779eb69fe93d050c16c769f069', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0'] >Thread-322::INFO::2014-05-13 11:14:53,694::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_imageNS already registered >Thread-322::INFO::2014-05-13 11:14:53,694::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_volumeNS already registered >Thread-322::INFO::2014-05-13 11:14:53,694::blockSD::452::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_lvmActivationNS already registered >Thread-322::DEBUG::2014-05-13 11:14:53,695::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 11:14:53,900::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:14:53,904::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:14:53,918::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00373907 s, 1.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:14:54,425::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 11:14:54,434::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:14:54,434::task::595::TaskManager.Task::(_updateState) Task=`e0a078ea-29e7-4eef-9b2a-d9e042399fb0`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:14:54,434::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:14:54,435::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:14:54,435::task::1185::TaskManager.Task::(prepare) Task=`e0a078ea-29e7-4eef-9b2a-d9e042399fb0`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:14:54,435::task::595::TaskManager.Task::(_updateState) Task=`e0a078ea-29e7-4eef-9b2a-d9e042399fb0`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:14:54,435::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:14:54,436::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:14:54,436::task::990::TaskManager.Task::(_decref) Task=`e0a078ea-29e7-4eef-9b2a-d9e042399fb0`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:14:54,442::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:14:54,443::task::595::TaskManager.Task::(_updateState) Task=`582fd0b7-9e52-4361-814a-6475111985a6`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:14:54,443::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:14:54,444::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`431524e2-3416-4bbb-8341-f5c906f4defc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:14:54,444::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:14:54,444::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:14:54,444::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`431524e2-3416-4bbb-8341-f5c906f4defc`::Granted request >Thread-338::DEBUG::2014-05-13 11:14:54,445::task::827::TaskManager.Task::(resourceAcquired) Task=`582fd0b7-9e52-4361-814a-6475111985a6`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:14:54,445::task::990::TaskManager.Task::(_decref) Task=`582fd0b7-9e52-4361-814a-6475111985a6`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:14:54,446::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:14:54,446::task::1185::TaskManager.Task::(prepare) Task=`582fd0b7-9e52-4361-814a-6475111985a6`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:14:54,446::task::595::TaskManager.Task::(_updateState) Task=`582fd0b7-9e52-4361-814a-6475111985a6`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:14:54,446::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:14:54,446::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:14:54,447::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:14:54,447::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:14:54,447::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:14:54,447::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:14:54,447::task::990::TaskManager.Task::(_decref) Task=`582fd0b7-9e52-4361-814a-6475111985a6`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:14:54,819::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.351474 s, 2.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:14:56,144::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:14:56,832::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:56,908::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313115 s, 32.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:14:58,921::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:14:59,003::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0386379 s, 26.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:14:59,163::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:15:01,016::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:01,097::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365478 s, 28.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:15:02,189::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:15:03,110::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:03,191::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383527 s, 26.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:15:03,925::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:15:03,944::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.0113135 s, 362 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:15:04,468::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:15:04,469::task::595::TaskManager.Task::(_updateState) Task=`fea8ffcb-e4f1-4c98-a09a-9d1b62e66d7f`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:15:04,469::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:15:04,469::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:15:04,469::task::1185::TaskManager.Task::(prepare) Task=`fea8ffcb-e4f1-4c98-a09a-9d1b62e66d7f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:15:04,470::task::595::TaskManager.Task::(_updateState) Task=`fea8ffcb-e4f1-4c98-a09a-9d1b62e66d7f`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:15:04,470::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:15:04,470::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:15:04,470::task::990::TaskManager.Task::(_decref) Task=`fea8ffcb-e4f1-4c98-a09a-9d1b62e66d7f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:15:04,475::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:15:04,475::task::595::TaskManager.Task::(_updateState) Task=`3b0a81df-c7a2-4406-bff3-38156e14f1f5`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:15:04,475::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:15:04,476::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c39fb94c-7fd3-4131-9964-5092948b3a99`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:15:04,476::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:15:04,476::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:15:04,476::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c39fb94c-7fd3-4131-9964-5092948b3a99`::Granted request >Thread-338::DEBUG::2014-05-13 11:15:04,477::task::827::TaskManager.Task::(resourceAcquired) Task=`3b0a81df-c7a2-4406-bff3-38156e14f1f5`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:15:04,477::task::990::TaskManager.Task::(_decref) Task=`3b0a81df-c7a2-4406-bff3-38156e14f1f5`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:15:04,478::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:15:04,478::task::1185::TaskManager.Task::(prepare) Task=`3b0a81df-c7a2-4406-bff3-38156e14f1f5`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:15:04,478::task::595::TaskManager.Task::(_updateState) Task=`3b0a81df-c7a2-4406-bff3-38156e14f1f5`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:15:04,478::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:15:04,478::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:15:04,479::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:15:04,479::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:15:04,479::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:15:04,480::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:15:04,480::task::990::TaskManager.Task::(_decref) Task=`3b0a81df-c7a2-4406-bff3-38156e14f1f5`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:15:05,204::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:15:05,213::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:15:05,295::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0454202 s, 22.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:15:05,799::task::595::TaskManager.Task::(_updateState) Task=`756aa799-79a2-4c8f-9058-9b407e6c1265`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:15:05,799::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:15:05,799::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0113135', 'lastCheck': '1.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:15:05,800::task::1185::TaskManager.Task::(prepare) Task=`756aa799-79a2-4c8f-9058-9b407e6c1265`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0113135', 'lastCheck': '1.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:15:05,800::task::595::TaskManager.Task::(_updateState) Task=`756aa799-79a2-4c8f-9058-9b407e6c1265`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:15:05,800::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:15:05,800::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:15:05,800::task::990::TaskManager.Task::(_decref) Task=`756aa799-79a2-4c8f-9058-9b407e6c1265`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:15:07,308::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:07,383::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314141 s, 32.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:15:08,230::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:15:09,396::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:09,483::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0444805 s, 23.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:15:11,249::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:15:11,496::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:11,570::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306672 s, 33.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:15:13,583::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:13,660::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.034586 s, 29.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:15:13,965::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:15:13,974::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000532782 s, 7.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:15:14,270::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:15:14,499::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:15:14,534::task::595::TaskManager.Task::(_updateState) Task=`c8d2f474-24e8-4fde-8dbc-933c2e1b9d21`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:15:14,534::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:15:14,535::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:15:14,535::task::1185::TaskManager.Task::(prepare) Task=`c8d2f474-24e8-4fde-8dbc-933c2e1b9d21`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:15:14,535::task::595::TaskManager.Task::(_updateState) Task=`c8d2f474-24e8-4fde-8dbc-933c2e1b9d21`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:15:14,535::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:15:14,535::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:15:14,536::task::990::TaskManager.Task::(_decref) Task=`c8d2f474-24e8-4fde-8dbc-933c2e1b9d21`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:15:14,542::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:15:14,542::task::595::TaskManager.Task::(_updateState) Task=`70bc58a3-3ba2-4806-82be-43354b27db2b`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:15:14,542::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:15:14,543::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5aedc8d8-4e12-4b29-b0d7-c3183987f27f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:15:14,543::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:15:14,543::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:15:14,543::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5aedc8d8-4e12-4b29-b0d7-c3183987f27f`::Granted request >Thread-338::DEBUG::2014-05-13 11:15:14,544::task::827::TaskManager.Task::(resourceAcquired) Task=`70bc58a3-3ba2-4806-82be-43354b27db2b`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:15:14,544::task::990::TaskManager.Task::(_decref) Task=`70bc58a3-3ba2-4806-82be-43354b27db2b`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:15:14,545::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:15:14,545::task::1185::TaskManager.Task::(prepare) Task=`70bc58a3-3ba2-4806-82be-43354b27db2b`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:15:14,545::task::595::TaskManager.Task::(_updateState) Task=`70bc58a3-3ba2-4806-82be-43354b27db2b`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:15:14,545::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:15:14,545::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:15:14,546::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:15:14,546::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:15:14,546::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:15:14,546::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:15:14,546::task::990::TaskManager.Task::(_decref) Task=`70bc58a3-3ba2-4806-82be-43354b27db2b`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:15:15,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:15,753::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0378792 s, 27.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:15:17,290::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:15:17,766::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:17,854::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0450256 s, 22.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:15:19,867::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:19,941::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303692 s, 33.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:15:20,330::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:15:20,936::task::595::TaskManager.Task::(_updateState) Task=`fb423e81-33b7-4a6a-ab96-9c7b8cc31646`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:15:20,937::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:15:20,937::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000532782', 'lastCheck': '7.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:15:20,937::task::1185::TaskManager.Task::(prepare) Task=`fb423e81-33b7-4a6a-ab96-9c7b8cc31646`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000532782', 'lastCheck': '7.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:15:20,937::task::595::TaskManager.Task::(_updateState) Task=`fb423e81-33b7-4a6a-ab96-9c7b8cc31646`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:15:20,937::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:15:20,938::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:15:20,938::task::990::TaskManager.Task::(_decref) Task=`fb423e81-33b7-4a6a-ab96-9c7b8cc31646`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:15:21,954::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:22,033::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373613 s, 27.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:15:23,348::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:15:23,981::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:15:23,989::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000619143 s, 6.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:15:24,046::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:24,128::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0396899 s, 25.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:15:24,564::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:15:24,565::task::595::TaskManager.Task::(_updateState) Task=`1c265200-c749-4452-97b6-2e8697dcb5b9`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:15:24,565::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:15:24,565::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:15:24,565::task::1185::TaskManager.Task::(prepare) Task=`1c265200-c749-4452-97b6-2e8697dcb5b9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:15:24,566::task::595::TaskManager.Task::(_updateState) Task=`1c265200-c749-4452-97b6-2e8697dcb5b9`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:15:24,566::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:15:24,566::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:15:24,566::task::990::TaskManager.Task::(_decref) Task=`1c265200-c749-4452-97b6-2e8697dcb5b9`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:15:24,571::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:15:24,571::task::595::TaskManager.Task::(_updateState) Task=`64e6d59a-3909-4bac-84c9-d52367e0ed17`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:15:24,571::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:15:24,572::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ff8c1bde-3a5f-4eeb-979a-f9ff3eb544dc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:15:24,572::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:15:24,572::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:15:24,572::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ff8c1bde-3a5f-4eeb-979a-f9ff3eb544dc`::Granted request >Thread-338::DEBUG::2014-05-13 11:15:24,573::task::827::TaskManager.Task::(resourceAcquired) Task=`64e6d59a-3909-4bac-84c9-d52367e0ed17`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:15:24,573::task::990::TaskManager.Task::(_decref) Task=`64e6d59a-3909-4bac-84c9-d52367e0ed17`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:15:24,574::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:15:24,574::task::1185::TaskManager.Task::(prepare) Task=`64e6d59a-3909-4bac-84c9-d52367e0ed17`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:15:24,574::task::595::TaskManager.Task::(_updateState) Task=`64e6d59a-3909-4bac-84c9-d52367e0ed17`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:15:24,574::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:15:24,574::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:15:24,575::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:15:24,575::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:15:24,575::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:15:24,575::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:15:24,575::task::990::TaskManager.Task::(_decref) Task=`64e6d59a-3909-4bac-84c9-d52367e0ed17`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:15:26,141::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:26,215::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0299889 s, 34.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:15:26,367::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:15:28,228::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:28,307::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0432524 s, 23.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:15:29,386::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:15:30,315::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:30,395::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0368196 s, 27.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:15:32,404::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:15:32,408::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:32,764::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.313752 s, 3.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:15:33,996::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:15:34,005::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000687056 s, 6.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:15:34,593::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:15:34,594::task::595::TaskManager.Task::(_updateState) Task=`953c1df8-76bd-4df3-9d85-3f0ce10e6558`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:15:34,594::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:15:34,595::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:15:34,595::task::1185::TaskManager.Task::(prepare) Task=`953c1df8-76bd-4df3-9d85-3f0ce10e6558`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:15:34,595::task::595::TaskManager.Task::(_updateState) Task=`953c1df8-76bd-4df3-9d85-3f0ce10e6558`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:15:34,595::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:15:34,595::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:15:34,595::task::990::TaskManager.Task::(_decref) Task=`953c1df8-76bd-4df3-9d85-3f0ce10e6558`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:15:34,600::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:15:34,601::task::595::TaskManager.Task::(_updateState) Task=`46560888-31ed-43c1-bef0-3790916f3637`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:15:34,601::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:15:34,601::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d7fb49f7-ccdf-4b23-8f8b-1d35c8314520`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:15:34,602::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:15:34,602::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:15:34,602::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d7fb49f7-ccdf-4b23-8f8b-1d35c8314520`::Granted request >Thread-338::DEBUG::2014-05-13 11:15:34,602::task::827::TaskManager.Task::(resourceAcquired) Task=`46560888-31ed-43c1-bef0-3790916f3637`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:15:34,602::task::990::TaskManager.Task::(_decref) Task=`46560888-31ed-43c1-bef0-3790916f3637`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:15:34,603::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:15:34,603::task::1185::TaskManager.Task::(prepare) Task=`46560888-31ed-43c1-bef0-3790916f3637`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:15:34,603::task::595::TaskManager.Task::(_updateState) Task=`46560888-31ed-43c1-bef0-3790916f3637`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:15:34,604::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:15:34,604::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:15:34,604::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:15:34,604::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:15:34,604::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:15:34,604::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:15:34,605::task::990::TaskManager.Task::(_decref) Task=`46560888-31ed-43c1-bef0-3790916f3637`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:15:34,777::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:34,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317601 s, 32.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:15:35,423::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:15:36,078::task::595::TaskManager.Task::(_updateState) Task=`719a9d9f-50dc-4072-9c36-2b812bc65f86`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:15:36,078::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:15:36,079::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000687056', 'lastCheck': '2.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:15:36,079::task::1185::TaskManager.Task::(prepare) Task=`719a9d9f-50dc-4072-9c36-2b812bc65f86`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000687056', 'lastCheck': '2.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:15:36,079::task::595::TaskManager.Task::(_updateState) Task=`719a9d9f-50dc-4072-9c36-2b812bc65f86`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:15:36,079::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:15:36,079::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:15:36,079::task::990::TaskManager.Task::(_decref) Task=`719a9d9f-50dc-4072-9c36-2b812bc65f86`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:15:36,864::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:36,943::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353178 s, 29.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:15:38,445::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:15:38,956::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:39,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310691 s, 33.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:15:41,045::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:41,125::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0377572 s, 27.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:15:41,463::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:15:43,138::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:43,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0375851 s, 27.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:15:44,011::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:15:44,020::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000668713 s, 6.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:15:44,482::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:15:44,625::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4478bd69] >Thread-66::DEBUG::2014-05-13 11:15:44,625::task::595::TaskManager.Task::(_updateState) Task=`a59cdd0a-c96a-4a10-8272-376894512c66`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:15:44,625::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:15:44,626::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:15:44,626::task::1185::TaskManager.Task::(prepare) Task=`a59cdd0a-c96a-4a10-8272-376894512c66`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:15:44,626::task::595::TaskManager.Task::(_updateState) Task=`a59cdd0a-c96a-4a10-8272-376894512c66`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:15:44,626::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:15:44,626::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:15:44,627::task::990::TaskManager.Task::(_decref) Task=`a59cdd0a-c96a-4a10-8272-376894512c66`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:15:44,631::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4478bd69] >Thread-338::DEBUG::2014-05-13 11:15:44,632::task::595::TaskManager.Task::(_updateState) Task=`9239f1d1-b169-472f-a001-4b03f9fa89ce`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:15:44,632::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:15:44,632::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0257e81a-1ab1-47e2-80c5-837d372cd62c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:15:44,633::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:15:44,633::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:15:44,633::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0257e81a-1ab1-47e2-80c5-837d372cd62c`::Granted request >Thread-338::DEBUG::2014-05-13 11:15:44,633::task::827::TaskManager.Task::(resourceAcquired) Task=`9239f1d1-b169-472f-a001-4b03f9fa89ce`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:15:44,633::task::990::TaskManager.Task::(_decref) Task=`9239f1d1-b169-472f-a001-4b03f9fa89ce`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:15:44,634::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:15:44,635::task::1185::TaskManager.Task::(prepare) Task=`9239f1d1-b169-472f-a001-4b03f9fa89ce`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:15:44,635::task::595::TaskManager.Task::(_updateState) Task=`9239f1d1-b169-472f-a001-4b03f9fa89ce`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:15:44,635::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:15:44,635::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:15:44,635::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:15:44,636::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:15:44,636::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:15:44,636::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:15:44,636::task::990::TaskManager.Task::(_decref) Task=`9239f1d1-b169-472f-a001-4b03f9fa89ce`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:15:45,231::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:45,600::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.326261 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:15:47,500::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:15:47,613::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:47,689::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322485 s, 31.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:15:49,703::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:49,777::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307077 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:15:50,519::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:15:51,216::task::595::TaskManager.Task::(_updateState) Task=`1891bb49-ed8a-410f-8a5f-487dbb4ae61d`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:15:51,217::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:15:51,217::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000668713', 'lastCheck': '7.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:15:51,217::task::1185::TaskManager.Task::(prepare) Task=`1891bb49-ed8a-410f-8a5f-487dbb4ae61d`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000668713', 'lastCheck': '7.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:15:51,217::task::595::TaskManager.Task::(_updateState) Task=`1891bb49-ed8a-410f-8a5f-487dbb4ae61d`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:15:51,217::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:15:51,218::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:15:51,218::task::990::TaskManager.Task::(_decref) Task=`1891bb49-ed8a-410f-8a5f-487dbb4ae61d`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:15:51,790::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:51,869::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0350929 s, 29.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:15:53,539::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:15:53,881::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:53,956::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320536 s, 31.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:15:54,048::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:15:54,070::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.0136202 s, 301 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:15:54,659::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17998964] >Thread-66::DEBUG::2014-05-13 11:15:54,659::task::595::TaskManager.Task::(_updateState) Task=`5b4c46e5-3af7-4f1b-99aa-33ed28125d56`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:15:54,660::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:15:54,660::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:15:54,660::task::1185::TaskManager.Task::(prepare) Task=`5b4c46e5-3af7-4f1b-99aa-33ed28125d56`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:15:54,660::task::595::TaskManager.Task::(_updateState) Task=`5b4c46e5-3af7-4f1b-99aa-33ed28125d56`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:15:54,660::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:15:54,661::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:15:54,661::task::990::TaskManager.Task::(_decref) Task=`5b4c46e5-3af7-4f1b-99aa-33ed28125d56`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:15:54,665::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17998964] >Thread-338::DEBUG::2014-05-13 11:15:54,666::task::595::TaskManager.Task::(_updateState) Task=`546f5046-9da0-4c6d-b436-d37adc541e35`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:15:54,666::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:15:54,666::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`943635f5-0850-4932-9dda-636d9d0e272c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:15:54,667::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:15:54,667::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:15:54,667::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`943635f5-0850-4932-9dda-636d9d0e272c`::Granted request >Thread-338::DEBUG::2014-05-13 11:15:54,667::task::827::TaskManager.Task::(resourceAcquired) Task=`546f5046-9da0-4c6d-b436-d37adc541e35`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:15:54,668::task::990::TaskManager.Task::(_decref) Task=`546f5046-9da0-4c6d-b436-d37adc541e35`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:15:54,668::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:15:54,668::task::1185::TaskManager.Task::(prepare) Task=`546f5046-9da0-4c6d-b436-d37adc541e35`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:15:54,669::task::595::TaskManager.Task::(_updateState) Task=`546f5046-9da0-4c6d-b436-d37adc541e35`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:15:54,669::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:15:54,669::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:15:54,669::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:15:54,669::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:15:54,670::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:15:54,670::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:15:54,670::task::990::TaskManager.Task::(_decref) Task=`546f5046-9da0-4c6d-b436-d37adc541e35`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:15:55,969::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:56,043::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309036 s, 33.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:15:56,567::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:15:58,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:15:58,150::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0503217 s, 20.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:15:59,587::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:16:00,163::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:00,242::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357079 s, 28.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:16:02,255::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:02,330::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.030337 s, 33.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:16:02,610::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:16:04,077::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:16:04,085::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000662746 s, 6.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:16:04,343::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:04,416::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0295156 s, 34.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:16:04,689::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:16:04,689::task::595::TaskManager.Task::(_updateState) Task=`ec38cefc-0a57-4375-ab59-36a9c23bba29`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:16:04,690::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:16:04,690::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:16:04,690::task::1185::TaskManager.Task::(prepare) Task=`ec38cefc-0a57-4375-ab59-36a9c23bba29`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:16:04,690::task::595::TaskManager.Task::(_updateState) Task=`ec38cefc-0a57-4375-ab59-36a9c23bba29`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:16:04,691::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:16:04,691::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:16:04,691::task::990::TaskManager.Task::(_decref) Task=`ec38cefc-0a57-4375-ab59-36a9c23bba29`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:16:04,696::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:16:04,696::task::595::TaskManager.Task::(_updateState) Task=`5a49b6d3-893e-4cea-a418-754eafc74847`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:16:04,696::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:16:04,697::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5a434824-afbd-4cef-9319-8766cf0c9ddd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:16:04,697::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:16:04,697::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:16:04,697::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5a434824-afbd-4cef-9319-8766cf0c9ddd`::Granted request >Thread-338::DEBUG::2014-05-13 11:16:04,698::task::827::TaskManager.Task::(resourceAcquired) Task=`5a49b6d3-893e-4cea-a418-754eafc74847`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:16:04,698::task::990::TaskManager.Task::(_decref) Task=`5a49b6d3-893e-4cea-a418-754eafc74847`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:16:04,699::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:16:04,699::task::1185::TaskManager.Task::(prepare) Task=`5a49b6d3-893e-4cea-a418-754eafc74847`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:16:04,699::task::595::TaskManager.Task::(_updateState) Task=`5a49b6d3-893e-4cea-a418-754eafc74847`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:16:04,699::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:16:04,699::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:16:04,700::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:16:04,700::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:16:04,700::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:16:04,700::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:16:04,700::task::990::TaskManager.Task::(_decref) Task=`5a49b6d3-893e-4cea-a418-754eafc74847`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:16:05,629::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:16:06,349::task::595::TaskManager.Task::(_updateState) Task=`7ce6a7bb-d230-4850-9b80-afdc0bd2e8e2`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:16:06,349::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:16:06,350::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000662746', 'lastCheck': '2.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:16:06,350::task::1185::TaskManager.Task::(prepare) Task=`7ce6a7bb-d230-4850-9b80-afdc0bd2e8e2`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000662746', 'lastCheck': '2.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:16:06,350::task::595::TaskManager.Task::(_updateState) Task=`7ce6a7bb-d230-4850-9b80-afdc0bd2e8e2`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:16:06,350::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:16:06,350::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:16:06,350::task::990::TaskManager.Task::(_decref) Task=`7ce6a7bb-d230-4850-9b80-afdc0bd2e8e2`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:16:06,430::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:06,513::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0393323 s, 26.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:16:08,526::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:08,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316824 s, 32.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:16:08,650::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:16:10,616::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:10,702::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0419424 s, 24.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:16:11,672::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:16:12,716::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:13,090::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.330009 s, 3.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:16:14,092::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:16:14,100::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000651455 s, 6.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:16:14,694::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:16:14,722::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [dc1f215] >Thread-66::DEBUG::2014-05-13 11:16:14,723::task::595::TaskManager.Task::(_updateState) Task=`9f670f76-f8ce-4fa0-8807-59e4347cac3d`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:16:14,723::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:16:14,723::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:16:14,723::task::1185::TaskManager.Task::(prepare) Task=`9f670f76-f8ce-4fa0-8807-59e4347cac3d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:16:14,724::task::595::TaskManager.Task::(_updateState) Task=`9f670f76-f8ce-4fa0-8807-59e4347cac3d`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:16:14,724::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:16:14,724::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:16:14,724::task::990::TaskManager.Task::(_decref) Task=`9f670f76-f8ce-4fa0-8807-59e4347cac3d`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:16:14,728::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [dc1f215] >Thread-338::DEBUG::2014-05-13 11:16:14,729::task::595::TaskManager.Task::(_updateState) Task=`3eef6890-56a4-40f7-9584-bc67137df014`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:16:14,729::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:16:14,729::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`de1cb8f9-d36f-41d3-b21b-cb2d2d92ec06`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:16:14,730::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:16:14,730::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:16:14,730::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`de1cb8f9-d36f-41d3-b21b-cb2d2d92ec06`::Granted request >Thread-338::DEBUG::2014-05-13 11:16:14,731::task::827::TaskManager.Task::(resourceAcquired) Task=`3eef6890-56a4-40f7-9584-bc67137df014`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:16:14,731::task::990::TaskManager.Task::(_decref) Task=`3eef6890-56a4-40f7-9584-bc67137df014`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:16:14,732::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:16:14,732::task::1185::TaskManager.Task::(prepare) Task=`3eef6890-56a4-40f7-9584-bc67137df014`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:16:14,732::task::595::TaskManager.Task::(_updateState) Task=`3eef6890-56a4-40f7-9584-bc67137df014`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:16:14,732::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:16:14,732::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:16:14,733::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:16:14,733::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:16:14,733::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:16:14,733::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:16:14,733::task::990::TaskManager.Task::(_decref) Task=`3eef6890-56a4-40f7-9584-bc67137df014`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:16:15,103::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:15,183::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364729 s, 28.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:16:17,196::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:17,270::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314274 s, 32.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:16:17,712::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:16:19,282::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:19,356::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0372024 s, 27.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:16:20,731::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:16:21,364::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:21,438::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307136 s, 33.3 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:16:21,490::task::595::TaskManager.Task::(_updateState) Task=`4ac098cd-a349-40d3-a661-00a93c11cdb3`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:16:21,490::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:16:21,491::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000651455', 'lastCheck': '7.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:16:21,491::task::1185::TaskManager.Task::(prepare) Task=`4ac098cd-a349-40d3-a661-00a93c11cdb3`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000651455', 'lastCheck': '7.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:16:21,491::task::595::TaskManager.Task::(_updateState) Task=`4ac098cd-a349-40d3-a661-00a93c11cdb3`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:16:21,491::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:16:21,491::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:16:21,491::task::990::TaskManager.Task::(_decref) Task=`4ac098cd-a349-40d3-a661-00a93c11cdb3`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:16:23,450::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:16:23,750::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:16:23,842::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.348509 s, 2.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:16:24,133::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:16:24,142::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000754488 s, 5.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:16:24,751::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3c71a9dd] >Thread-66::DEBUG::2014-05-13 11:16:24,752::task::595::TaskManager.Task::(_updateState) Task=`c2ca50bc-5f9a-453e-948f-cceb1dff2d5f`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:16:24,752::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:16:24,753::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:16:24,753::task::1185::TaskManager.Task::(prepare) Task=`c2ca50bc-5f9a-453e-948f-cceb1dff2d5f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:16:24,753::task::595::TaskManager.Task::(_updateState) Task=`c2ca50bc-5f9a-453e-948f-cceb1dff2d5f`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:16:24,753::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:16:24,753::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:16:24,754::task::990::TaskManager.Task::(_decref) Task=`c2ca50bc-5f9a-453e-948f-cceb1dff2d5f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:16:24,758::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3c71a9dd] >Thread-338::DEBUG::2014-05-13 11:16:24,758::task::595::TaskManager.Task::(_updateState) Task=`505ac626-9927-4016-a320-32d6614957ee`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:16:24,759::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:16:24,759::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`373ae1fc-49d5-4eff-8ae8-826e72b0dc10`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:16:24,759::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:16:24,760::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:16:24,760::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`373ae1fc-49d5-4eff-8ae8-826e72b0dc10`::Granted request >Thread-338::DEBUG::2014-05-13 11:16:24,760::task::827::TaskManager.Task::(resourceAcquired) Task=`505ac626-9927-4016-a320-32d6614957ee`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:16:24,760::task::990::TaskManager.Task::(_decref) Task=`505ac626-9927-4016-a320-32d6614957ee`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:16:24,761::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:16:24,761::task::1185::TaskManager.Task::(prepare) Task=`505ac626-9927-4016-a320-32d6614957ee`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:16:24,761::task::595::TaskManager.Task::(_updateState) Task=`505ac626-9927-4016-a320-32d6614957ee`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:16:24,762::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:16:24,762::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:16:24,762::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:16:24,762::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:16:24,762::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:16:24,762::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:16:24,763::task::990::TaskManager.Task::(_decref) Task=`505ac626-9927-4016-a320-32d6614957ee`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:16:25,855::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:25,928::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305453 s, 33.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:16:26,774::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:16:27,941::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:28,019::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.03577 s, 28.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:16:29,793::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:16:30,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:30,111::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358379 s, 28.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:16:32,124::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:32,198::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0304658 s, 33.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:16:32,810::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:16:34,149::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:16:34,157::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000587887 s, 7.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:16:34,211::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:34,291::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360974 s, 28.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:16:34,783::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [50b954f9] >Thread-66::DEBUG::2014-05-13 11:16:34,783::task::595::TaskManager.Task::(_updateState) Task=`de1c7b45-76de-47c1-a380-936510583db4`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:16:34,784::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:16:34,784::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:16:34,784::task::1185::TaskManager.Task::(prepare) Task=`de1c7b45-76de-47c1-a380-936510583db4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:16:34,784::task::595::TaskManager.Task::(_updateState) Task=`de1c7b45-76de-47c1-a380-936510583db4`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:16:34,784::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:16:34,785::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:16:34,785::task::990::TaskManager.Task::(_decref) Task=`de1c7b45-76de-47c1-a380-936510583db4`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:16:34,789::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [50b954f9] >Thread-338::DEBUG::2014-05-13 11:16:34,790::task::595::TaskManager.Task::(_updateState) Task=`7390d26c-8833-4f77-b181-e42e263f9e28`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:16:34,790::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:16:34,790::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1713ad3a-e66f-468c-9525-13e23d45d552`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:16:34,790::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:16:34,791::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:16:34,791::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1713ad3a-e66f-468c-9525-13e23d45d552`::Granted request >Thread-338::DEBUG::2014-05-13 11:16:34,791::task::827::TaskManager.Task::(resourceAcquired) Task=`7390d26c-8833-4f77-b181-e42e263f9e28`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:16:34,791::task::990::TaskManager.Task::(_decref) Task=`7390d26c-8833-4f77-b181-e42e263f9e28`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:16:34,792::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:16:34,792::task::1185::TaskManager.Task::(prepare) Task=`7390d26c-8833-4f77-b181-e42e263f9e28`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:16:34,793::task::595::TaskManager.Task::(_updateState) Task=`7390d26c-8833-4f77-b181-e42e263f9e28`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:16:34,793::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:16:34,793::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:16:34,793::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:16:34,793::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:16:34,794::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:16:34,794::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:16:34,794::task::990::TaskManager.Task::(_decref) Task=`7390d26c-8833-4f77-b181-e42e263f9e28`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:16:35,833::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:16:36,304::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:36,384::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356948 s, 28.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:16:36,626::task::595::TaskManager.Task::(_updateState) Task=`109772b6-058c-42fd-b743-d3b06a4d3a7c`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:16:36,626::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:16:36,627::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000587887', 'lastCheck': '2.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:16:36,627::task::1185::TaskManager.Task::(prepare) Task=`109772b6-058c-42fd-b743-d3b06a4d3a7c`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000587887', 'lastCheck': '2.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:16:36,627::task::595::TaskManager.Task::(_updateState) Task=`109772b6-058c-42fd-b743-d3b06a4d3a7c`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:16:36,627::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:16:36,627::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:16:36,627::task::990::TaskManager.Task::(_decref) Task=`109772b6-058c-42fd-b743-d3b06a4d3a7c`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:16:38,397::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:38,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.410055 s, 2.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:16:38,856::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:16:40,866::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:40,940::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0301036 s, 34.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:16:41,875::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:16:42,953::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:43,033::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0376002 s, 27.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:16:44,164::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:16:44,173::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000601492 s, 6.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:16:44,815::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:16:44,815::task::595::TaskManager.Task::(_updateState) Task=`381c3653-6bcc-4d0e-aba5-ea9d96729ed8`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:16:44,816::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:16:44,816::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:16:44,816::task::1185::TaskManager.Task::(prepare) Task=`381c3653-6bcc-4d0e-aba5-ea9d96729ed8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:16:44,816::task::595::TaskManager.Task::(_updateState) Task=`381c3653-6bcc-4d0e-aba5-ea9d96729ed8`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:16:44,816::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:16:44,817::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:16:44,817::task::990::TaskManager.Task::(_decref) Task=`381c3653-6bcc-4d0e-aba5-ea9d96729ed8`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:16:44,821::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:16:44,821::task::595::TaskManager.Task::(_updateState) Task=`87369c34-9b34-4a9f-ba5a-a0d8cd469390`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:16:44,822::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:16:44,822::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fcb00992-699d-4b89-95d7-25fea546f438`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:16:44,822::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:16:44,823::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:16:44,823::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fcb00992-699d-4b89-95d7-25fea546f438`::Granted request >Thread-338::DEBUG::2014-05-13 11:16:44,823::task::827::TaskManager.Task::(resourceAcquired) Task=`87369c34-9b34-4a9f-ba5a-a0d8cd469390`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:16:44,823::task::990::TaskManager.Task::(_decref) Task=`87369c34-9b34-4a9f-ba5a-a0d8cd469390`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:16:44,824::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:16:44,824::task::1185::TaskManager.Task::(prepare) Task=`87369c34-9b34-4a9f-ba5a-a0d8cd469390`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:16:44,825::task::595::TaskManager.Task::(_updateState) Task=`87369c34-9b34-4a9f-ba5a-a0d8cd469390`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:16:44,825::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:16:44,825::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:16:44,825::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:16:44,825::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:16:44,825::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:16:44,826::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:16:44,826::task::990::TaskManager.Task::(_decref) Task=`87369c34-9b34-4a9f-ba5a-a0d8cd469390`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:16:44,893::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:16:45,046::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:45,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358952 s, 28.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:16:47,140::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:47,216::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0331564 s, 30.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:16:47,912::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:16:49,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:49,306::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0344626 s, 29.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:16:50,931::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:16:51,319::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:51,414::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.051837 s, 19.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:16:51,758::task::595::TaskManager.Task::(_updateState) Task=`a3974f9a-e64b-41ec-842b-1cbf57860d51`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:16:51,758::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:16:51,759::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000601492', 'lastCheck': '7.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:16:51,759::task::1185::TaskManager.Task::(prepare) Task=`a3974f9a-e64b-41ec-842b-1cbf57860d51`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000601492', 'lastCheck': '7.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:16:51,759::task::595::TaskManager.Task::(_updateState) Task=`a3974f9a-e64b-41ec-842b-1cbf57860d51`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:16:51,759::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:16:51,759::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:16:51,760::task::990::TaskManager.Task::(_decref) Task=`a3974f9a-e64b-41ec-842b-1cbf57860d51`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:16:53,461::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:53,533::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0296483 s, 34.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:16:53,950::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:16:54,181::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:16:54,189::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000735748 s, 5.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:16:54,846::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:16:54,846::task::595::TaskManager.Task::(_updateState) Task=`0e6c9ae5-eeda-4cf6-9b78-e57088ad0688`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:16:54,847::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:16:54,847::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:16:54,847::task::1185::TaskManager.Task::(prepare) Task=`0e6c9ae5-eeda-4cf6-9b78-e57088ad0688`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:16:54,847::task::595::TaskManager.Task::(_updateState) Task=`0e6c9ae5-eeda-4cf6-9b78-e57088ad0688`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:16:54,847::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:16:54,848::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:16:54,848::task::990::TaskManager.Task::(_decref) Task=`0e6c9ae5-eeda-4cf6-9b78-e57088ad0688`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:16:54,852::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:16:54,853::task::595::TaskManager.Task::(_updateState) Task=`2b585d98-989e-4606-af1b-12ae2408f590`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:16:54,853::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:16:54,853::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`32e7cbd0-0c0f-4a68-8952-d04210871cec`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:16:54,853::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:16:54,854::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:16:54,854::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`32e7cbd0-0c0f-4a68-8952-d04210871cec`::Granted request >Thread-338::DEBUG::2014-05-13 11:16:54,854::task::827::TaskManager.Task::(resourceAcquired) Task=`2b585d98-989e-4606-af1b-12ae2408f590`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:16:54,854::task::990::TaskManager.Task::(_decref) Task=`2b585d98-989e-4606-af1b-12ae2408f590`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:16:54,855::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:16:54,855::task::1185::TaskManager.Task::(prepare) Task=`2b585d98-989e-4606-af1b-12ae2408f590`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:16:54,855::task::595::TaskManager.Task::(_updateState) Task=`2b585d98-989e-4606-af1b-12ae2408f590`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:16:54,856::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:16:54,856::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:16:54,856::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:16:54,856::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:16:54,856::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:16:54,857::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:16:54,857::task::990::TaskManager.Task::(_decref) Task=`2b585d98-989e-4606-af1b-12ae2408f590`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:16:55,546::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:55,621::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305488 s, 33.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:16:56,970::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:16:57,634::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:57,725::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0488289 s, 21.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:16:59,738::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:16:59,819::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362567 s, 28.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:16:59,990::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:17:01,832::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:17:01,908::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313904 s, 32.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:17:03,009::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:17:03,921::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:17:04,002::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0386499 s, 26.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:17:04,196::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:17:04,206::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00216784 s, 1.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:17:04,876::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:17:04,876::task::595::TaskManager.Task::(_updateState) Task=`fc960b73-8477-46d6-a2c9-3571c85ec5f0`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:17:04,877::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:17:04,877::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:17:04,877::task::1185::TaskManager.Task::(prepare) Task=`fc960b73-8477-46d6-a2c9-3571c85ec5f0`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:17:04,877::task::595::TaskManager.Task::(_updateState) Task=`fc960b73-8477-46d6-a2c9-3571c85ec5f0`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:17:04,878::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:17:04,878::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:17:04,878::task::990::TaskManager.Task::(_decref) Task=`fc960b73-8477-46d6-a2c9-3571c85ec5f0`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:17:04,882::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:17:04,883::task::595::TaskManager.Task::(_updateState) Task=`90db1ffe-6bd0-4c83-a7ab-0aa626cb8753`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:17:04,883::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:17:04,883::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fab1d471-b9e4-4b32-9dd0-7608c3fe1d12`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:17:04,884::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:17:04,884::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:17:04,884::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fab1d471-b9e4-4b32-9dd0-7608c3fe1d12`::Granted request >Thread-338::DEBUG::2014-05-13 11:17:04,884::task::827::TaskManager.Task::(resourceAcquired) Task=`90db1ffe-6bd0-4c83-a7ab-0aa626cb8753`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:17:04,885::task::990::TaskManager.Task::(_decref) Task=`90db1ffe-6bd0-4c83-a7ab-0aa626cb8753`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:17:04,885::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:17:04,885::task::1185::TaskManager.Task::(prepare) Task=`90db1ffe-6bd0-4c83-a7ab-0aa626cb8753`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:17:04,886::task::595::TaskManager.Task::(_updateState) Task=`90db1ffe-6bd0-4c83-a7ab-0aa626cb8753`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:17:04,886::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:17:04,886::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:17:04,886::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:17:04,886::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:17:04,887::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:17:04,887::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:17:04,887::task::990::TaskManager.Task::(_decref) Task=`90db1ffe-6bd0-4c83-a7ab-0aa626cb8753`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:17:06,016::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:17:06,032::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:17:06,107::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0500042 s, 20.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:17:06,889::task::595::TaskManager.Task::(_updateState) Task=`70f097d7-4300-4d5e-b837-7719ca1e9ff0`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:17:06,889::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:17:06,890::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00216784', 'lastCheck': '2.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:17:06,890::task::1185::TaskManager.Task::(prepare) Task=`70f097d7-4300-4d5e-b837-7719ca1e9ff0`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00216784', 'lastCheck': '2.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:17:06,890::task::595::TaskManager.Task::(_updateState) Task=`70f097d7-4300-4d5e-b837-7719ca1e9ff0`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:17:06,890::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:17:06,890::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:17:06,891::task::990::TaskManager.Task::(_decref) Task=`70f097d7-4300-4d5e-b837-7719ca1e9ff0`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:17:08,120::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:17:08,194::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312341 s, 32.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:17:09,052::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:17:10,208::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:17:10,295::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0446034 s, 23.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:17:12,071::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:17:12,308::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:17:12,388::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359763 s, 28.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:17:14,213::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:17:14,229::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00859933 s, 476 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:17:14,401::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:17:14,480::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0375227 s, 27.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:17:14,906::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3256e43f] >Thread-13::DEBUG::2014-05-13 11:17:14,907::task::595::TaskManager.Task::(_updateState) Task=`4145e0fd-edfc-42d0-8e89-1c3dfb5681ca`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:17:14,907::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:17:14,907::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:17:14,907::task::1185::TaskManager.Task::(prepare) Task=`4145e0fd-edfc-42d0-8e89-1c3dfb5681ca`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:17:14,908::task::595::TaskManager.Task::(_updateState) Task=`4145e0fd-edfc-42d0-8e89-1c3dfb5681ca`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:17:14,908::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:17:14,908::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:17:14,908::task::990::TaskManager.Task::(_decref) Task=`4145e0fd-edfc-42d0-8e89-1c3dfb5681ca`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:17:14,913::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3256e43f] >Thread-338::DEBUG::2014-05-13 11:17:14,913::task::595::TaskManager.Task::(_updateState) Task=`16c11b8e-abb1-4352-a101-a2c5370351cf`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:17:14,913::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:17:14,914::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8a9819b2-070a-4326-9955-30839f61528a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:17:14,914::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:17:14,914::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:17:14,914::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8a9819b2-070a-4326-9955-30839f61528a`::Granted request >Thread-338::DEBUG::2014-05-13 11:17:14,915::task::827::TaskManager.Task::(resourceAcquired) Task=`16c11b8e-abb1-4352-a101-a2c5370351cf`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:17:14,915::task::990::TaskManager.Task::(_decref) Task=`16c11b8e-abb1-4352-a101-a2c5370351cf`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:17:14,916::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:17:14,916::task::1185::TaskManager.Task::(prepare) Task=`16c11b8e-abb1-4352-a101-a2c5370351cf`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:17:14,916::task::595::TaskManager.Task::(_updateState) Task=`16c11b8e-abb1-4352-a101-a2c5370351cf`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:17:14,916::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:17:14,916::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:17:14,917::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:17:14,917::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:17:14,917::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:17:14,917::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:17:14,917::task::990::TaskManager.Task::(_decref) Task=`16c11b8e-abb1-4352-a101-a2c5370351cf`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:17:15,095::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:17:16,494::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:17:16,568::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.032146 s, 31.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:17:18,116::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:17:18,581::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:17:18,664::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0400725 s, 25.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:17:20,677::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:17:21,043::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.323258 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:17:21,135::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:17:22,042::task::595::TaskManager.Task::(_updateState) Task=`e64307a1-b84e-47f6-b088-6b58221c0047`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:17:22,042::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:17:22,043::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00859933', 'lastCheck': '7.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:17:22,043::task::1185::TaskManager.Task::(prepare) Task=`e64307a1-b84e-47f6-b088-6b58221c0047`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00859933', 'lastCheck': '7.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:17:22,043::task::595::TaskManager.Task::(_updateState) Task=`e64307a1-b84e-47f6-b088-6b58221c0047`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:17:22,043::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:17:22,043::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:17:22,044::task::990::TaskManager.Task::(_decref) Task=`e64307a1-b84e-47f6-b088-6b58221c0047`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:17:23,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:17:23,131::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313165 s, 32.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:17:24,155::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:17:24,236::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:17:24,244::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000591741 s, 6.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:17:24,937::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:17:24,938::task::595::TaskManager.Task::(_updateState) Task=`c56caf63-cb37-4dfb-945c-17657ca0efc3`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:17:24,938::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:17:24,939::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:17:24,939::task::1185::TaskManager.Task::(prepare) Task=`c56caf63-cb37-4dfb-945c-17657ca0efc3`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:17:24,939::task::595::TaskManager.Task::(_updateState) Task=`c56caf63-cb37-4dfb-945c-17657ca0efc3`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:17:24,939::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:17:24,939::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:17:24,939::task::990::TaskManager.Task::(_decref) Task=`c56caf63-cb37-4dfb-945c-17657ca0efc3`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:17:24,944::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:17:24,944::task::595::TaskManager.Task::(_updateState) Task=`f3a23c65-9340-4077-bfdd-da54610d6def`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:17:24,945::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:17:24,945::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1d8ea3d6-413e-4a99-91e4-56bbcb816f24`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:17:24,945::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:17:24,946::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:17:24,946::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1d8ea3d6-413e-4a99-91e4-56bbcb816f24`::Granted request >Thread-338::DEBUG::2014-05-13 11:17:24,946::task::827::TaskManager.Task::(resourceAcquired) Task=`f3a23c65-9340-4077-bfdd-da54610d6def`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:17:24,946::task::990::TaskManager.Task::(_decref) Task=`f3a23c65-9340-4077-bfdd-da54610d6def`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:17:24,947::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:17:24,947::task::1185::TaskManager.Task::(prepare) Task=`f3a23c65-9340-4077-bfdd-da54610d6def`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:17:24,947::task::595::TaskManager.Task::(_updateState) Task=`f3a23c65-9340-4077-bfdd-da54610d6def`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:17:24,948::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:17:24,948::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:17:24,948::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:17:24,948::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:17:24,948::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:17:24,948::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:17:24,949::task::990::TaskManager.Task::(_decref) Task=`f3a23c65-9340-4077-bfdd-da54610d6def`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:17:25,144::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:17:25,227::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387778 s, 26.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:17:27,174::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:17:27,240::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:17:27,316::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313183 s, 32.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:17:29,329::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:17:29,404::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0298383 s, 34.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:17:30,209::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:17:31,417::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:17:31,500::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0406483 s, 25.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:17:33,232::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:17:33,513::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:17:33,881::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.32571 s, 3.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:17:34,251::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:17:34,268::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00872518 s, 469 kB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:17:34,967::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [303c64a0] >Thread-13::DEBUG::2014-05-13 11:17:34,968::task::595::TaskManager.Task::(_updateState) Task=`13c460c8-0235-40e7-a437-6c09af062f55`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:17:34,968::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:17:34,969::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:17:34,969::task::1185::TaskManager.Task::(prepare) Task=`13c460c8-0235-40e7-a437-6c09af062f55`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:17:34,969::task::595::TaskManager.Task::(_updateState) Task=`13c460c8-0235-40e7-a437-6c09af062f55`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:17:34,969::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:17:34,969::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:17:34,969::task::990::TaskManager.Task::(_decref) Task=`13c460c8-0235-40e7-a437-6c09af062f55`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:17:35,275::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [303c64a0] >Thread-338::DEBUG::2014-05-13 11:17:35,275::task::595::TaskManager.Task::(_updateState) Task=`8474f38d-37ad-416a-a69c-2a50cbcb35da`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:17:35,276::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:17:35,277::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`18417be0-3564-45b3-9f82-7fe3d427382e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:17:35,277::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:17:35,278::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:17:35,278::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`18417be0-3564-45b3-9f82-7fe3d427382e`::Granted request >Thread-338::DEBUG::2014-05-13 11:17:35,278::task::827::TaskManager.Task::(resourceAcquired) Task=`8474f38d-37ad-416a-a69c-2a50cbcb35da`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:17:35,278::task::990::TaskManager.Task::(_decref) Task=`8474f38d-37ad-416a-a69c-2a50cbcb35da`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:17:35,279::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:17:35,279::task::1185::TaskManager.Task::(prepare) Task=`8474f38d-37ad-416a-a69c-2a50cbcb35da`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:17:35,279::task::595::TaskManager.Task::(_updateState) Task=`8474f38d-37ad-416a-a69c-2a50cbcb35da`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:17:35,280::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:17:35,280::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:17:35,280::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:17:35,280::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:17:35,280::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:17:35,280::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:17:35,281::task::990::TaskManager.Task::(_decref) Task=`8474f38d-37ad-416a-a69c-2a50cbcb35da`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:17:35,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:17:35,973::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0349039 s, 29.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:17:36,267::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:17:37,181::task::595::TaskManager.Task::(_updateState) Task=`55291db5-4f76-434a-b23f-fb9354b6a972`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:17:37,181::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:17:37,181::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00872518', 'lastCheck': '2.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:17:37,181::task::1185::TaskManager.Task::(prepare) Task=`55291db5-4f76-434a-b23f-fb9354b6a972`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00872518', 'lastCheck': '2.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:17:37,181::task::595::TaskManager.Task::(_updateState) Task=`55291db5-4f76-434a-b23f-fb9354b6a972`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:17:37,182::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:17:37,182::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:17:37,182::task::990::TaskManager.Task::(_decref) Task=`55291db5-4f76-434a-b23f-fb9354b6a972`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:17:37,986::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:17:38,059::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0299078 s, 34.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:17:39,288::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:17:40,073::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:17:40,155::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379606 s, 27.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:17:42,168::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:17:42,246::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0337939 s, 30.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:17:42,306::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:17:44,259::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 11:17:44,275::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:17:44,301::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000791842 s, 5.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:17:44,376::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0751515 s, 13.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:17:45,301::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [45b9a91f] >Thread-13::DEBUG::2014-05-13 11:17:45,302::task::595::TaskManager.Task::(_updateState) Task=`b5c699d6-375c-4663-a53a-46691497f529`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:17:45,302::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:17:45,303::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:17:45,303::task::1185::TaskManager.Task::(prepare) Task=`b5c699d6-375c-4663-a53a-46691497f529`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:17:45,303::task::595::TaskManager.Task::(_updateState) Task=`b5c699d6-375c-4663-a53a-46691497f529`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:17:45,303::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:17:45,303::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:17:45,303::task::990::TaskManager.Task::(_decref) Task=`b5c699d6-375c-4663-a53a-46691497f529`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:17:45,308::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [45b9a91f] >Thread-338::DEBUG::2014-05-13 11:17:45,308::task::595::TaskManager.Task::(_updateState) Task=`6c879aed-69df-4aca-ab1d-982e20998ea0`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:17:45,308::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:17:45,309::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`cdaa824a-aae4-485b-bebc-914f481f6a1a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:17:45,309::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:17:45,309::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:17:45,309::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`cdaa824a-aae4-485b-bebc-914f481f6a1a`::Granted request >Thread-338::DEBUG::2014-05-13 11:17:45,310::task::827::TaskManager.Task::(resourceAcquired) Task=`6c879aed-69df-4aca-ab1d-982e20998ea0`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:17:45,310::task::990::TaskManager.Task::(_decref) Task=`6c879aed-69df-4aca-ab1d-982e20998ea0`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:17:45,311::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:17:45,311::task::1185::TaskManager.Task::(prepare) Task=`6c879aed-69df-4aca-ab1d-982e20998ea0`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:17:45,311::task::595::TaskManager.Task::(_updateState) Task=`6c879aed-69df-4aca-ab1d-982e20998ea0`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:17:45,311::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:17:45,312::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:17:45,312::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:17:45,312::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:17:45,312::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:17:45,312::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:17:45,313::task::990::TaskManager.Task::(_decref) Task=`6c879aed-69df-4aca-ab1d-982e20998ea0`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:17:45,323::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:17:46,389::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:17:46,478::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0466417 s, 22.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:17:48,342::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:17:48,491::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:17:48,576::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0375509 s, 27.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:17:50,594::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:17:50,677::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0386394 s, 26.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:17:51,369::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:17:52,317::task::595::TaskManager.Task::(_updateState) Task=`e49f72d2-dab9-461b-9c84-5497f31e6011`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:17:52,317::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:17:52,317::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000791842', 'lastCheck': '8.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:17:52,317::task::1185::TaskManager.Task::(prepare) Task=`e49f72d2-dab9-461b-9c84-5497f31e6011`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000791842', 'lastCheck': '8.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:17:52,317::task::595::TaskManager.Task::(_updateState) Task=`e49f72d2-dab9-461b-9c84-5497f31e6011`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:17:52,318::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:17:52,318::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:17:52,318::task::990::TaskManager.Task::(_decref) Task=`e49f72d2-dab9-461b-9c84-5497f31e6011`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:17:52,690::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:17:52,764::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306982 s, 33.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:17:54,330::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:17:54,338::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000643995 s, 6.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:17:54,393::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:17:54,777::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:17:54,856::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0354425 s, 28.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:17:55,331::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:17:55,332::task::595::TaskManager.Task::(_updateState) Task=`54ff6011-aae0-4bd8-8399-3c81a024de69`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:17:55,332::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:17:55,332::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:17:55,332::task::1185::TaskManager.Task::(prepare) Task=`54ff6011-aae0-4bd8-8399-3c81a024de69`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:17:55,332::task::595::TaskManager.Task::(_updateState) Task=`54ff6011-aae0-4bd8-8399-3c81a024de69`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:17:55,333::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:17:55,333::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:17:55,333::task::990::TaskManager.Task::(_decref) Task=`54ff6011-aae0-4bd8-8399-3c81a024de69`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:17:55,337::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:17:55,338::task::595::TaskManager.Task::(_updateState) Task=`92f81f82-a4e2-4c93-a736-fb5719df6ee0`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:17:55,338::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:17:55,338::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e8410a26-cee7-4761-8b9a-bb4945857dff`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:17:55,338::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:17:55,339::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:17:55,339::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e8410a26-cee7-4761-8b9a-bb4945857dff`::Granted request >Thread-338::DEBUG::2014-05-13 11:17:55,339::task::827::TaskManager.Task::(resourceAcquired) Task=`92f81f82-a4e2-4c93-a736-fb5719df6ee0`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:17:55,339::task::990::TaskManager.Task::(_decref) Task=`92f81f82-a4e2-4c93-a736-fb5719df6ee0`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:17:55,340::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:17:55,340::task::1185::TaskManager.Task::(prepare) Task=`92f81f82-a4e2-4c93-a736-fb5719df6ee0`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:17:55,341::task::595::TaskManager.Task::(_updateState) Task=`92f81f82-a4e2-4c93-a736-fb5719df6ee0`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:17:55,341::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:17:55,341::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:17:55,341::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:17:55,341::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:17:55,341::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:17:55,342::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:17:55,342::task::990::TaskManager.Task::(_decref) Task=`92f81f82-a4e2-4c93-a736-fb5719df6ee0`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:17:56,869::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:17:56,962::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306876 s, 33.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:17:57,411::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:17:58,975::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:17:59,051::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309681 s, 33.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:18:00,428::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:18:01,063::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:01,445::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.334046 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:18:03,451::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:18:03,453::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:03,527::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307062 s, 33.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:18:04,345::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:18:04,354::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000808643 s, 5.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:18:05,361::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:18:05,362::task::595::TaskManager.Task::(_updateState) Task=`a815f251-acc0-44f3-89fa-d5f6378c14f3`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:18:05,362::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:18:05,363::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:18:05,363::task::1185::TaskManager.Task::(prepare) Task=`a815f251-acc0-44f3-89fa-d5f6378c14f3`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:18:05,363::task::595::TaskManager.Task::(_updateState) Task=`a815f251-acc0-44f3-89fa-d5f6378c14f3`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:18:05,363::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:18:05,363::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:18:05,363::task::990::TaskManager.Task::(_decref) Task=`a815f251-acc0-44f3-89fa-d5f6378c14f3`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:18:05,368::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:18:05,368::task::595::TaskManager.Task::(_updateState) Task=`ffacf29c-f950-4374-8c2e-558df0257672`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:18:05,368::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:18:05,369::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1c377241-7024-4470-99ca-5983bf817fe6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:18:05,369::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:18:05,369::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:18:05,369::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1c377241-7024-4470-99ca-5983bf817fe6`::Granted request >Thread-338::DEBUG::2014-05-13 11:18:05,370::task::827::TaskManager.Task::(resourceAcquired) Task=`ffacf29c-f950-4374-8c2e-558df0257672`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:18:05,370::task::990::TaskManager.Task::(_decref) Task=`ffacf29c-f950-4374-8c2e-558df0257672`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:18:05,371::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:18:05,371::task::1185::TaskManager.Task::(prepare) Task=`ffacf29c-f950-4374-8c2e-558df0257672`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:18:05,371::task::595::TaskManager.Task::(_updateState) Task=`ffacf29c-f950-4374-8c2e-558df0257672`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:18:05,371::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:18:05,371::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:18:05,372::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:18:05,372::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:18:05,372::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:18:05,372::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:18:05,372::task::990::TaskManager.Task::(_decref) Task=`ffacf29c-f950-4374-8c2e-558df0257672`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:18:05,540::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:05,620::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360021 s, 28.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:18:06,468::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:18:07,454::task::595::TaskManager.Task::(_updateState) Task=`123af677-ed89-40f3-b62e-97399c122813`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:18:07,454::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:18:07,454::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000808643', 'lastCheck': '3.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:18:07,454::task::1185::TaskManager.Task::(prepare) Task=`123af677-ed89-40f3-b62e-97399c122813`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000808643', 'lastCheck': '3.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:18:07,454::task::595::TaskManager.Task::(_updateState) Task=`123af677-ed89-40f3-b62e-97399c122813`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:18:07,454::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:18:07,455::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:18:07,455::task::990::TaskManager.Task::(_decref) Task=`123af677-ed89-40f3-b62e-97399c122813`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:18:07,633::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:07,710::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321171 s, 31.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:18:09,487::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:18:09,723::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:09,804::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0385652 s, 26.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:18:11,817::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:11,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316646 s, 32.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:18:12,506::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:18:13,908::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:14,277::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.323593 s, 3.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:18:14,361::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:18:14,370::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000792686 s, 5.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:18:15,391::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:18:15,427::task::595::TaskManager.Task::(_updateState) Task=`8522bfbd-93ee-4d74-9858-56867fa3a4ba`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:18:15,427::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:18:15,428::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:18:15,428::task::1185::TaskManager.Task::(prepare) Task=`8522bfbd-93ee-4d74-9858-56867fa3a4ba`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:18:15,428::task::595::TaskManager.Task::(_updateState) Task=`8522bfbd-93ee-4d74-9858-56867fa3a4ba`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:18:15,428::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:18:15,428::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:18:15,429::task::990::TaskManager.Task::(_decref) Task=`8522bfbd-93ee-4d74-9858-56867fa3a4ba`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:18:15,434::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:18:15,434::task::595::TaskManager.Task::(_updateState) Task=`fc0f0599-26d4-4b65-8237-e70c44fe2806`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:18:15,434::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:18:15,435::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e7a03323-2869-4bc5-864f-1479ac6e6db4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:18:15,435::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:18:15,435::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:18:15,435::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e7a03323-2869-4bc5-864f-1479ac6e6db4`::Granted request >Thread-338::DEBUG::2014-05-13 11:18:15,436::task::827::TaskManager.Task::(resourceAcquired) Task=`fc0f0599-26d4-4b65-8237-e70c44fe2806`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:18:15,436::task::990::TaskManager.Task::(_decref) Task=`fc0f0599-26d4-4b65-8237-e70c44fe2806`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:18:15,436::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:18:15,437::task::1185::TaskManager.Task::(prepare) Task=`fc0f0599-26d4-4b65-8237-e70c44fe2806`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:18:15,437::task::595::TaskManager.Task::(_updateState) Task=`fc0f0599-26d4-4b65-8237-e70c44fe2806`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:18:15,437::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:18:15,437::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:18:15,437::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:18:15,438::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:18:15,438::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:18:15,438::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:18:15,438::task::990::TaskManager.Task::(_decref) Task=`fc0f0599-26d4-4b65-8237-e70c44fe2806`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:18:15,525::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:18:16,290::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:16,368::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356987 s, 28.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:18:18,381::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:18,457::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310085 s, 33.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:18:18,543::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:18:20,470::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:20,548::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362168 s, 28.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:18:21,563::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:18:22,561::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 11:18:22,585::task::595::TaskManager.Task::(_updateState) Task=`04711e56-4d59-4ad8-a309-334e2fc17fab`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:18:22,586::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:18:22,586::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000792686', 'lastCheck': '8.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:18:22,586::task::1185::TaskManager.Task::(prepare) Task=`04711e56-4d59-4ad8-a309-334e2fc17fab`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000792686', 'lastCheck': '8.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:18:22,587::task::595::TaskManager.Task::(_updateState) Task=`04711e56-4d59-4ad8-a309-334e2fc17fab`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:18:22,587::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:18:22,587::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:18:22,587::task::990::TaskManager.Task::(_decref) Task=`04711e56-4d59-4ad8-a309-334e2fc17fab`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:18:22,645::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0375576 s, 27.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:18:24,377::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:18:24,386::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000736213 s, 5.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:18:24,582::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:18:24,658::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:24,738::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359819 s, 28.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:18:25,458::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5eff0b16] >Thread-13::DEBUG::2014-05-13 11:18:25,459::task::595::TaskManager.Task::(_updateState) Task=`1b9f6c81-6d8f-4008-9ef8-b1a7c8f86411`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:18:25,459::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:18:25,459::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:18:25,459::task::1185::TaskManager.Task::(prepare) Task=`1b9f6c81-6d8f-4008-9ef8-b1a7c8f86411`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:18:25,460::task::595::TaskManager.Task::(_updateState) Task=`1b9f6c81-6d8f-4008-9ef8-b1a7c8f86411`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:18:25,460::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:18:25,460::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:18:25,460::task::990::TaskManager.Task::(_decref) Task=`1b9f6c81-6d8f-4008-9ef8-b1a7c8f86411`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:18:25,465::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5eff0b16] >Thread-338::DEBUG::2014-05-13 11:18:25,465::task::595::TaskManager.Task::(_updateState) Task=`d83d9e68-2fb7-42b2-ad14-7dbcb76f64ac`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:18:25,465::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:18:25,465::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4ff7a32d-985a-437e-955e-25a83005113f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:18:25,466::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:18:25,466::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:18:25,466::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4ff7a32d-985a-437e-955e-25a83005113f`::Granted request >Thread-338::DEBUG::2014-05-13 11:18:25,466::task::827::TaskManager.Task::(resourceAcquired) Task=`d83d9e68-2fb7-42b2-ad14-7dbcb76f64ac`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:18:25,466::task::990::TaskManager.Task::(_decref) Task=`d83d9e68-2fb7-42b2-ad14-7dbcb76f64ac`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:18:25,467::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:18:25,467::task::1185::TaskManager.Task::(prepare) Task=`d83d9e68-2fb7-42b2-ad14-7dbcb76f64ac`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:18:25,467::task::595::TaskManager.Task::(_updateState) Task=`d83d9e68-2fb7-42b2-ad14-7dbcb76f64ac`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:18:25,468::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:18:25,468::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:18:25,468::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:18:25,468::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:18:25,468::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:18:25,468::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:18:25,469::task::990::TaskManager.Task::(_decref) Task=`d83d9e68-2fb7-42b2-ad14-7dbcb76f64ac`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:18:26,751::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:26,839::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0466461 s, 22.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:18:27,605::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:18:28,852::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:28,927::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0328523 s, 31.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:18:30,624::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:18:30,940::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:31,031::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0366927 s, 27.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:18:33,044::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:33,119::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.032873 s, 31.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:18:33,642::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:18:34,393::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:18:34,402::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000769716 s, 5.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:18:35,132::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:35,212::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0368159 s, 27.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:18:35,490::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:18:35,490::task::595::TaskManager.Task::(_updateState) Task=`66440554-a95c-404c-81f4-3ed88978176f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:18:35,491::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:18:35,491::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:18:35,491::task::1185::TaskManager.Task::(prepare) Task=`66440554-a95c-404c-81f4-3ed88978176f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:18:35,491::task::595::TaskManager.Task::(_updateState) Task=`66440554-a95c-404c-81f4-3ed88978176f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:18:35,491::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:18:35,492::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:18:35,492::task::990::TaskManager.Task::(_decref) Task=`66440554-a95c-404c-81f4-3ed88978176f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:18:35,497::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:18:35,497::task::595::TaskManager.Task::(_updateState) Task=`f3f687f2-5c52-4295-b17c-6f883744f186`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:18:35,497::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:18:35,497::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`14a366b6-82c8-49f2-8f50-dab8ba4e10a8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:18:35,498::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:18:35,498::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:18:35,498::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`14a366b6-82c8-49f2-8f50-dab8ba4e10a8`::Granted request >Thread-338::DEBUG::2014-05-13 11:18:35,498::task::827::TaskManager.Task::(resourceAcquired) Task=`f3f687f2-5c52-4295-b17c-6f883744f186`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:18:35,499::task::990::TaskManager.Task::(_decref) Task=`f3f687f2-5c52-4295-b17c-6f883744f186`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:18:35,499::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:18:35,500::task::1185::TaskManager.Task::(prepare) Task=`f3f687f2-5c52-4295-b17c-6f883744f186`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:18:35,500::task::595::TaskManager.Task::(_updateState) Task=`f3f687f2-5c52-4295-b17c-6f883744f186`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:18:35,500::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:18:35,500::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:18:35,500::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:18:35,501::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:18:35,501::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:18:35,501::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:18:35,501::task::990::TaskManager.Task::(_decref) Task=`f3f687f2-5c52-4295-b17c-6f883744f186`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:18:36,661::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:18:37,224::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:37,305::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383703 s, 26.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:18:37,720::task::595::TaskManager.Task::(_updateState) Task=`42cb041a-3ab9-4428-ba84-5bf286ebeeaa`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:18:37,720::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:18:37,720::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000769716', 'lastCheck': '3.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:18:37,721::task::1185::TaskManager.Task::(prepare) Task=`42cb041a-3ab9-4428-ba84-5bf286ebeeaa`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000769716', 'lastCheck': '3.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:18:37,721::task::595::TaskManager.Task::(_updateState) Task=`42cb041a-3ab9-4428-ba84-5bf286ebeeaa`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:18:37,721::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:18:37,721::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:18:37,721::task::990::TaskManager.Task::(_decref) Task=`42cb041a-3ab9-4428-ba84-5bf286ebeeaa`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:18:39,318::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:39,392::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321422 s, 31.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:18:39,680::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:18:41,406::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:41,762::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.314042 s, 3.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:18:42,699::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:18:43,775::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:43,855::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313316 s, 32.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:18:44,409::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:18:44,418::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000611279 s, 6.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:18:45,522::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:18:45,523::task::595::TaskManager.Task::(_updateState) Task=`1b2e6bd3-5b63-43c0-8086-00e7149a5ad6`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:18:45,523::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:18:45,523::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:18:45,523::task::1185::TaskManager.Task::(prepare) Task=`1b2e6bd3-5b63-43c0-8086-00e7149a5ad6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:18:45,524::task::595::TaskManager.Task::(_updateState) Task=`1b2e6bd3-5b63-43c0-8086-00e7149a5ad6`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:18:45,524::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:18:45,524::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:18:45,524::task::990::TaskManager.Task::(_decref) Task=`1b2e6bd3-5b63-43c0-8086-00e7149a5ad6`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:18:45,529::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:18:45,529::task::595::TaskManager.Task::(_updateState) Task=`42f0842d-2749-47d3-905c-3b19ba7438a0`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:18:45,529::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:18:45,530::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2c11b7ba-86ca-41cb-8f11-60b1a5a957fd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:18:45,530::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:18:45,530::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:18:45,530::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2c11b7ba-86ca-41cb-8f11-60b1a5a957fd`::Granted request >Thread-338::DEBUG::2014-05-13 11:18:45,531::task::827::TaskManager.Task::(resourceAcquired) Task=`42f0842d-2749-47d3-905c-3b19ba7438a0`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:18:45,532::task::990::TaskManager.Task::(_decref) Task=`42f0842d-2749-47d3-905c-3b19ba7438a0`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:18:45,532::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:18:45,532::task::1185::TaskManager.Task::(prepare) Task=`42f0842d-2749-47d3-905c-3b19ba7438a0`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:18:45,533::task::595::TaskManager.Task::(_updateState) Task=`42f0842d-2749-47d3-905c-3b19ba7438a0`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:18:45,533::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:18:45,533::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:18:45,533::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:18:45,533::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:18:45,534::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:18:45,534::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:18:45,534::task::990::TaskManager.Task::(_decref) Task=`42f0842d-2749-47d3-905c-3b19ba7438a0`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:18:45,723::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:18:45,868::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:45,949::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383731 s, 26.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:18:47,961::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:48,037::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312503 s, 32.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:18:48,743::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:18:50,050::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:50,133::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037629 s, 27.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:18:51,767::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:18:52,146::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:52,227::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.03736 s, 27.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:18:52,863::task::595::TaskManager.Task::(_updateState) Task=`8ffdb07b-6350-450e-a1a6-e87bdb1f08ff`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:18:52,863::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:18:52,863::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000611279', 'lastCheck': '8.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:18:52,864::task::1185::TaskManager.Task::(prepare) Task=`8ffdb07b-6350-450e-a1a6-e87bdb1f08ff`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000611279', 'lastCheck': '8.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:18:52,864::task::595::TaskManager.Task::(_updateState) Task=`8ffdb07b-6350-450e-a1a6-e87bdb1f08ff`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:18:52,864::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:18:52,864::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:18:52,864::task::990::TaskManager.Task::(_decref) Task=`8ffdb07b-6350-450e-a1a6-e87bdb1f08ff`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:18:54,240::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:54,326::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0426024 s, 24.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:18:54,441::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:18:54,456::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00684123 s, 599 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:18:54,786::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:18:55,552::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:18:55,552::task::595::TaskManager.Task::(_updateState) Task=`815bf794-098e-4152-9796-8f8f94a5f4ab`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:18:55,552::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:18:55,553::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:18:55,553::task::1185::TaskManager.Task::(prepare) Task=`815bf794-098e-4152-9796-8f8f94a5f4ab`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:18:55,553::task::595::TaskManager.Task::(_updateState) Task=`815bf794-098e-4152-9796-8f8f94a5f4ab`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:18:55,553::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:18:55,553::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:18:55,554::task::990::TaskManager.Task::(_decref) Task=`815bf794-098e-4152-9796-8f8f94a5f4ab`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:18:55,558::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:18:55,558::task::595::TaskManager.Task::(_updateState) Task=`a58fd574-1582-4af6-9af9-7d0c76faf7ed`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:18:55,558::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:18:55,559::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0449e61b-7a50-4b11-b7df-4faf0b4954e7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:18:55,559::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:18:55,559::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:18:55,559::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0449e61b-7a50-4b11-b7df-4faf0b4954e7`::Granted request >Thread-338::DEBUG::2014-05-13 11:18:55,560::task::827::TaskManager.Task::(resourceAcquired) Task=`a58fd574-1582-4af6-9af9-7d0c76faf7ed`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:18:55,560::task::990::TaskManager.Task::(_decref) Task=`a58fd574-1582-4af6-9af9-7d0c76faf7ed`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:18:55,560::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:18:55,561::task::1185::TaskManager.Task::(prepare) Task=`a58fd574-1582-4af6-9af9-7d0c76faf7ed`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:18:55,561::task::595::TaskManager.Task::(_updateState) Task=`a58fd574-1582-4af6-9af9-7d0c76faf7ed`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:18:55,561::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:18:55,561::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:18:55,561::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:18:55,561::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:18:55,562::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:18:55,562::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:18:55,562::task::990::TaskManager.Task::(_decref) Task=`a58fd574-1582-4af6-9af9-7d0c76faf7ed`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:18:56,340::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:56,416::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309598 s, 33.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:18:57,804::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:18:58,429::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:18:58,506::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360216 s, 28.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:19:00,514::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:00,596::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379338 s, 27.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:19:00,826::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:19:02,609::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:02,683::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307427 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:19:03,848::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:19:04,463::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:19:04,471::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000664503 s, 6.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:19:04,696::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:04,772::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031199 s, 32.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:19:05,580::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [303c64a0] >Thread-13::DEBUG::2014-05-13 11:19:05,580::task::595::TaskManager.Task::(_updateState) Task=`dfe11c30-aec5-4ca7-8314-665c554abf58`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:19:05,581::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:19:05,581::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:19:05,581::task::1185::TaskManager.Task::(prepare) Task=`dfe11c30-aec5-4ca7-8314-665c554abf58`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:19:05,581::task::595::TaskManager.Task::(_updateState) Task=`dfe11c30-aec5-4ca7-8314-665c554abf58`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:19:05,582::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:19:05,582::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:19:05,582::task::990::TaskManager.Task::(_decref) Task=`dfe11c30-aec5-4ca7-8314-665c554abf58`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:19:05,586::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [303c64a0] >Thread-338::DEBUG::2014-05-13 11:19:05,587::task::595::TaskManager.Task::(_updateState) Task=`a7894f7b-51bf-4661-9b7c-3246da1e64e6`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:19:05,587::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:19:05,587::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`44d747cb-5311-4657-a8dc-4b8ab716ea86`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:19:05,587::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:19:05,588::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:19:05,588::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`44d747cb-5311-4657-a8dc-4b8ab716ea86`::Granted request >Thread-338::DEBUG::2014-05-13 11:19:05,588::task::827::TaskManager.Task::(resourceAcquired) Task=`a7894f7b-51bf-4661-9b7c-3246da1e64e6`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:19:05,588::task::990::TaskManager.Task::(_decref) Task=`a7894f7b-51bf-4661-9b7c-3246da1e64e6`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:19:05,589::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:19:05,589::task::1185::TaskManager.Task::(prepare) Task=`a7894f7b-51bf-4661-9b7c-3246da1e64e6`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:19:05,589::task::595::TaskManager.Task::(_updateState) Task=`a7894f7b-51bf-4661-9b7c-3246da1e64e6`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:19:05,590::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:19:05,590::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:19:05,590::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:19:05,590::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:19:05,590::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:19:05,590::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:19:05,591::task::990::TaskManager.Task::(_decref) Task=`a7894f7b-51bf-4661-9b7c-3246da1e64e6`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:19:06,785::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:19:06,866::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:19:07,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.340463 s, 3.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:19:07,994::task::595::TaskManager.Task::(_updateState) Task=`1aba3eb6-622d-4d68-a689-a9dd3a3fbbca`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:19:07,994::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:19:07,995::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000664503', 'lastCheck': '3.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:19:07,995::task::1185::TaskManager.Task::(prepare) Task=`1aba3eb6-622d-4d68-a689-a9dd3a3fbbca`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000664503', 'lastCheck': '3.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:19:07,995::task::595::TaskManager.Task::(_updateState) Task=`1aba3eb6-622d-4d68-a689-a9dd3a3fbbca`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:19:07,995::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:19:07,995::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:19:07,995::task::990::TaskManager.Task::(_decref) Task=`1aba3eb6-622d-4d68-a689-a9dd3a3fbbca`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:19:09,181::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:09,257::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312914 s, 32.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:19:09,885::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:19:11,270::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:11,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303714 s, 33.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:19:12,909::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:19:13,356::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:13,437::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0366029 s, 28.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:19:14,478::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:19:14,487::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000710136 s, 5.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:19:15,451::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:15,528::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0345107 s, 29.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:19:15,610::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:19:15,610::task::595::TaskManager.Task::(_updateState) Task=`fcc4732c-f5b8-40e0-9550-1b834fb607f2`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:19:15,610::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:19:15,611::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:19:15,611::task::1185::TaskManager.Task::(prepare) Task=`fcc4732c-f5b8-40e0-9550-1b834fb607f2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:19:15,611::task::595::TaskManager.Task::(_updateState) Task=`fcc4732c-f5b8-40e0-9550-1b834fb607f2`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:19:15,611::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:19:15,611::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:19:15,612::task::990::TaskManager.Task::(_decref) Task=`fcc4732c-f5b8-40e0-9550-1b834fb607f2`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:19:15,616::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:19:15,616::task::595::TaskManager.Task::(_updateState) Task=`f5c59094-338f-4d1c-8d93-7689cfe025d1`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:19:15,616::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:19:15,617::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c3214c46-8570-4018-ac56-cb8cdad005f7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:19:15,617::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:19:15,617::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:19:15,617::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c3214c46-8570-4018-ac56-cb8cdad005f7`::Granted request >Thread-338::DEBUG::2014-05-13 11:19:15,618::task::827::TaskManager.Task::(resourceAcquired) Task=`f5c59094-338f-4d1c-8d93-7689cfe025d1`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:19:15,618::task::990::TaskManager.Task::(_decref) Task=`f5c59094-338f-4d1c-8d93-7689cfe025d1`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:19:15,619::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:19:15,619::task::1185::TaskManager.Task::(prepare) Task=`f5c59094-338f-4d1c-8d93-7689cfe025d1`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:19:15,619::task::595::TaskManager.Task::(_updateState) Task=`f5c59094-338f-4d1c-8d93-7689cfe025d1`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:19:15,619::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:19:15,620::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:19:15,620::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:19:15,620::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:19:15,620::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:19:15,620::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:19:15,621::task::990::TaskManager.Task::(_decref) Task=`f5c59094-338f-4d1c-8d93-7689cfe025d1`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:19:15,928::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:19:17,541::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:17,615::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0302549 s, 33.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:19:18,948::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:19:19,628::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:19,717::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0457908 s, 22.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:19:21,730::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:21,809::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0366292 s, 28.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:19:21,968::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:19:23,126::task::595::TaskManager.Task::(_updateState) Task=`57c305c3-0097-426d-8343-d00bd110957a`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:19:23,126::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:19:23,126::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000710136', 'lastCheck': '8.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:19:23,126::task::1185::TaskManager.Task::(prepare) Task=`57c305c3-0097-426d-8343-d00bd110957a`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000710136', 'lastCheck': '8.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:19:23,126::task::595::TaskManager.Task::(_updateState) Task=`57c305c3-0097-426d-8343-d00bd110957a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:19:23,127::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:19:23,127::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:19:23,127::task::990::TaskManager.Task::(_decref) Task=`57c305c3-0097-426d-8343-d00bd110957a`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:19:23,822::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:23,898::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0335765 s, 30.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:19:24,494::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:19:24,503::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000564805 s, 7.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:19:24,987::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:19:25,639::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:19:25,640::task::595::TaskManager.Task::(_updateState) Task=`a1c51e52-1fb5-489a-ac8f-d0b70ecd6ddd`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:19:25,640::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:19:25,640::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:19:25,641::task::1185::TaskManager.Task::(prepare) Task=`a1c51e52-1fb5-489a-ac8f-d0b70ecd6ddd`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:19:25,641::task::595::TaskManager.Task::(_updateState) Task=`a1c51e52-1fb5-489a-ac8f-d0b70ecd6ddd`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:19:25,641::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:19:25,641::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:19:25,641::task::990::TaskManager.Task::(_decref) Task=`a1c51e52-1fb5-489a-ac8f-d0b70ecd6ddd`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:19:25,646::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:19:25,646::task::595::TaskManager.Task::(_updateState) Task=`247a8a6c-84b9-401d-8de7-1ea16c0d428e`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:19:25,646::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:19:25,647::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0c0f9b42-905c-428d-b84c-f92cd6086160`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:19:25,647::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:19:25,647::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:19:25,647::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0c0f9b42-905c-428d-b84c-f92cd6086160`::Granted request >Thread-338::DEBUG::2014-05-13 11:19:25,648::task::827::TaskManager.Task::(resourceAcquired) Task=`247a8a6c-84b9-401d-8de7-1ea16c0d428e`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:19:25,648::task::990::TaskManager.Task::(_decref) Task=`247a8a6c-84b9-401d-8de7-1ea16c0d428e`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:19:25,648::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:19:25,649::task::1185::TaskManager.Task::(prepare) Task=`247a8a6c-84b9-401d-8de7-1ea16c0d428e`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:19:25,649::task::595::TaskManager.Task::(_updateState) Task=`247a8a6c-84b9-401d-8de7-1ea16c0d428e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:19:25,649::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:19:25,649::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:19:25,649::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:19:25,649::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:19:25,650::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:19:25,650::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:19:25,650::task::990::TaskManager.Task::(_decref) Task=`247a8a6c-84b9-401d-8de7-1ea16c0d428e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:19:25,911::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:25,988::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316077 s, 32.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:19:28,001::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:19:28,013::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:19:28,080::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360183 s, 28.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:19:30,093::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:30,195::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0448372 s, 22.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:19:31,029::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:19:32,213::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:32,302::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0451317 s, 22.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:19:34,049::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:19:34,315::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:34,396::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0367398 s, 27.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:19:34,510::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:19:34,518::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000687092 s, 6.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:19:35,669::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [12ac301a] >Thread-13::DEBUG::2014-05-13 11:19:35,669::task::595::TaskManager.Task::(_updateState) Task=`c342c490-c6e1-477c-a1af-3c2b43bbe882`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:19:35,670::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:19:35,670::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:19:35,670::task::1185::TaskManager.Task::(prepare) Task=`c342c490-c6e1-477c-a1af-3c2b43bbe882`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:19:35,670::task::595::TaskManager.Task::(_updateState) Task=`c342c490-c6e1-477c-a1af-3c2b43bbe882`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:19:35,671::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:19:35,671::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:19:35,671::task::990::TaskManager.Task::(_decref) Task=`c342c490-c6e1-477c-a1af-3c2b43bbe882`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:19:35,675::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [12ac301a] >Thread-338::DEBUG::2014-05-13 11:19:35,676::task::595::TaskManager.Task::(_updateState) Task=`68db8fa5-305b-489a-9d0c-8f3226d15828`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:19:35,676::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:19:35,676::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`80f05a32-af8d-4365-94f3-2a97eabe5334`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:19:35,677::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:19:35,677::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:19:35,677::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`80f05a32-af8d-4365-94f3-2a97eabe5334`::Granted request >Thread-338::DEBUG::2014-05-13 11:19:35,677::task::827::TaskManager.Task::(resourceAcquired) Task=`68db8fa5-305b-489a-9d0c-8f3226d15828`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:19:35,678::task::990::TaskManager.Task::(_decref) Task=`68db8fa5-305b-489a-9d0c-8f3226d15828`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:19:35,678::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:19:35,678::task::1185::TaskManager.Task::(prepare) Task=`68db8fa5-305b-489a-9d0c-8f3226d15828`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:19:35,679::task::595::TaskManager.Task::(_updateState) Task=`68db8fa5-305b-489a-9d0c-8f3226d15828`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:19:35,679::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:19:35,679::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:19:35,679::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:19:35,679::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:19:35,680::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:19:35,680::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:19:35,680::task::990::TaskManager.Task::(_decref) Task=`68db8fa5-305b-489a-9d0c-8f3226d15828`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:19:36,409::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:36,490::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374402 s, 27.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:19:37,069::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:19:38,256::task::595::TaskManager.Task::(_updateState) Task=`5ba60e52-397c-4437-b2b8-a62594ea32ab`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:19:38,257::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:19:38,257::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000687092', 'lastCheck': '3.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:19:38,257::task::1185::TaskManager.Task::(prepare) Task=`5ba60e52-397c-4437-b2b8-a62594ea32ab`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000687092', 'lastCheck': '3.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:19:38,257::task::595::TaskManager.Task::(_updateState) Task=`5ba60e52-397c-4437-b2b8-a62594ea32ab`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:19:38,257::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:19:38,257::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:19:38,258::task::990::TaskManager.Task::(_decref) Task=`5ba60e52-397c-4437-b2b8-a62594ea32ab`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:19:38,502::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:38,576::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031105 s, 32.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:19:40,088::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:19:40,589::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:40,664::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306889 s, 33.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:19:42,677::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:42,755::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353486 s, 29.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:19:43,107::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:19:44,525::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:19:44,535::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00152419 s, 2.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:19:44,768::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:44,847::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360307 s, 28.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:19:45,698::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:19:45,698::task::595::TaskManager.Task::(_updateState) Task=`86c9dea2-5d89-4811-a822-d961c4d73c9a`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:19:45,699::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:19:45,699::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:19:45,699::task::1185::TaskManager.Task::(prepare) Task=`86c9dea2-5d89-4811-a822-d961c4d73c9a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:19:45,699::task::595::TaskManager.Task::(_updateState) Task=`86c9dea2-5d89-4811-a822-d961c4d73c9a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:19:45,699::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:19:45,700::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:19:45,700::task::990::TaskManager.Task::(_decref) Task=`86c9dea2-5d89-4811-a822-d961c4d73c9a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:19:45,704::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:19:45,704::task::595::TaskManager.Task::(_updateState) Task=`cfbe5102-93eb-419f-b144-50aa5c40e52a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:19:45,705::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:19:45,705::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4f5d9919-68eb-478a-968f-bcaed55487f2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:19:45,705::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:19:45,705::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:19:45,706::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4f5d9919-68eb-478a-968f-bcaed55487f2`::Granted request >Thread-338::DEBUG::2014-05-13 11:19:45,706::task::827::TaskManager.Task::(resourceAcquired) Task=`cfbe5102-93eb-419f-b144-50aa5c40e52a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:19:45,706::task::990::TaskManager.Task::(_decref) Task=`cfbe5102-93eb-419f-b144-50aa5c40e52a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:19:45,707::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:19:45,707::task::1185::TaskManager.Task::(prepare) Task=`cfbe5102-93eb-419f-b144-50aa5c40e52a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:19:45,707::task::595::TaskManager.Task::(_updateState) Task=`cfbe5102-93eb-419f-b144-50aa5c40e52a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:19:45,708::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:19:45,708::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:19:45,708::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:19:45,708::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:19:45,708::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:19:45,708::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:19:45,709::task::990::TaskManager.Task::(_decref) Task=`cfbe5102-93eb-419f-b144-50aa5c40e52a`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:19:46,125::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:19:46,860::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:46,949::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0463188 s, 22.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:19:48,962::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:49,043::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0375663 s, 27.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:19:49,144::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:19:51,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:51,138::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382222 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:19:52,162::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:19:53,152::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:53,228::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314697 s, 32.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:19:53,387::task::595::TaskManager.Task::(_updateState) Task=`bfc2e1b5-e403-4133-8d5f-ba14162e6f18`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:19:53,388::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:19:53,388::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00152419', 'lastCheck': '8.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:19:53,388::task::1185::TaskManager.Task::(prepare) Task=`bfc2e1b5-e403-4133-8d5f-ba14162e6f18`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00152419', 'lastCheck': '8.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:19:53,388::task::595::TaskManager.Task::(_updateState) Task=`bfc2e1b5-e403-4133-8d5f-ba14162e6f18`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:19:53,388::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:19:53,389::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:19:53,389::task::990::TaskManager.Task::(_decref) Task=`bfc2e1b5-e403-4133-8d5f-ba14162e6f18`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 11:19:54,539::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 41830dad-f012-4b4d-b636-863f66d5ca4b >Thread-322::DEBUG::2014-05-13 11:19:54,540::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-322::DEBUG::2014-05-13 11:19:54,542::blockSD::336::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:19:54,551::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000661585 s, 3.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:19:54,551::misc::287::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000661585 s, 3.1 MB/s'], size: 2048 >Thread-322::DEBUG::2014-05-13 11:19:54,552::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-322::WARNING::2014-05-13 11:19:54,552::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-322::DEBUG::2014-05-13 11:19:54,552::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-322::DEBUG::2014-05-13 11:19:54,552::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:19:54,553::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:19:54,553::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:19:54,553::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:19:54,553::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:19:54,554::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 11:19:54,774::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:19:54,776::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:19:54,776::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'LOCKPOLICY=', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', 'DESCRIPTION=ISCSI', 'LOGBLKSIZE=512', 'TYPE=ISCSI', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'PHYBLKSIZE=512', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', 'CLASS=Data', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'POOL_DESCRIPTION=DC_2', 'POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'MASTER_VERSION=1', 'ROLE=Master', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', '_SHA_CKSUM=28bb2d7cc60e0d779eb69fe93d050c16c769f069', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0'] >Thread-322::INFO::2014-05-13 11:19:54,778::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_imageNS already registered >Thread-322::INFO::2014-05-13 11:19:54,778::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_volumeNS already registered >Thread-322::INFO::2014-05-13 11:19:54,778::blockSD::452::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_lvmActivationNS already registered >Thread-322::DEBUG::2014-05-13 11:19:54,779::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 11:19:54,948::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:19:54,951::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:19:54,959::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000687462 s, 6.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:19:55,194::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:19:55,254::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:55,335::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365881 s, 28.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:19:55,732::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5eff0b16] >Thread-13::DEBUG::2014-05-13 11:19:55,733::task::595::TaskManager.Task::(_updateState) Task=`e24b72eb-508d-441c-8900-e3baf52bc03c`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:19:55,733::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:19:55,733::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:19:55,734::task::1185::TaskManager.Task::(prepare) Task=`e24b72eb-508d-441c-8900-e3baf52bc03c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:19:55,734::task::595::TaskManager.Task::(_updateState) Task=`e24b72eb-508d-441c-8900-e3baf52bc03c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:19:55,734::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:19:55,734::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:19:55,734::task::990::TaskManager.Task::(_decref) Task=`e24b72eb-508d-441c-8900-e3baf52bc03c`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:19:55,738::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5eff0b16] >Thread-338::DEBUG::2014-05-13 11:19:55,738::task::595::TaskManager.Task::(_updateState) Task=`84259909-172d-494a-ae83-df3112e821f4`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:19:55,739::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:19:55,739::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`948b0e98-4832-497c-8f90-a6c7ad465bd1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:19:55,739::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:19:55,739::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:19:55,740::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`948b0e98-4832-497c-8f90-a6c7ad465bd1`::Granted request >Thread-338::DEBUG::2014-05-13 11:19:55,740::task::827::TaskManager.Task::(resourceAcquired) Task=`84259909-172d-494a-ae83-df3112e821f4`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:19:55,740::task::990::TaskManager.Task::(_decref) Task=`84259909-172d-494a-ae83-df3112e821f4`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:19:55,741::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:19:55,741::task::1185::TaskManager.Task::(prepare) Task=`84259909-172d-494a-ae83-df3112e821f4`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:19:55,741::task::595::TaskManager.Task::(_updateState) Task=`84259909-172d-494a-ae83-df3112e821f4`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:19:55,741::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:19:55,741::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:19:55,741::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:19:55,742::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:19:55,742::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:19:55,742::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:19:55,742::task::990::TaskManager.Task::(_decref) Task=`84259909-172d-494a-ae83-df3112e821f4`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:19:57,348::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:57,423::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307221 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:19:58,216::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:19:59,435::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:19:59,513::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311117 s, 32.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:20:01,236::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:20:01,526::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:20:01,615::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0452698 s, 22.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:20:03,628::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:20:03,708::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362796 s, 28.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:20:04,254::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:20:04,966::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:20:04,975::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000709587 s, 5.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:20:05,721::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 11:20:05,759::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:20:05,761::task::595::TaskManager.Task::(_updateState) Task=`3b2097ca-07df-45d9-a476-4b6f57a605ee`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:20:05,762::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:20:05,764::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:20:05,765::task::1185::TaskManager.Task::(prepare) Task=`3b2097ca-07df-45d9-a476-4b6f57a605ee`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:20:05,766::task::595::TaskManager.Task::(_updateState) Task=`3b2097ca-07df-45d9-a476-4b6f57a605ee`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:20:05,767::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:20:05,768::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:20:05,769::task::990::TaskManager.Task::(_decref) Task=`3b2097ca-07df-45d9-a476-4b6f57a605ee`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:20:05,776::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:20:05,778::task::595::TaskManager.Task::(_updateState) Task=`a8e97628-264c-4636-945b-64f69fd29f6f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:20:05,779::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:20:05,781::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2850426d-1390-4ed1-86d6-8fa779c88f5c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:20:05,782::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:20:05,783::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:20:05,784::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2850426d-1390-4ed1-86d6-8fa779c88f5c`::Granted request >Thread-338::DEBUG::2014-05-13 11:20:05,786::task::827::TaskManager.Task::(resourceAcquired) Task=`a8e97628-264c-4636-945b-64f69fd29f6f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:20:05,787::task::990::TaskManager.Task::(_decref) Task=`a8e97628-264c-4636-945b-64f69fd29f6f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:20:05,792::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:20:05,793::task::1185::TaskManager.Task::(prepare) Task=`a8e97628-264c-4636-945b-64f69fd29f6f`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:20:05,794::task::595::TaskManager.Task::(_updateState) Task=`a8e97628-264c-4636-945b-64f69fd29f6f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:20:05,794::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:20:05,795::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:20:05,797::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:20:05,798::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:20:05,799::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:20:05,800::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:20:05,802::task::990::TaskManager.Task::(_decref) Task=`a8e97628-264c-4636-945b-64f69fd29f6f`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:20:05,839::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0568616 s, 18.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:20:07,275::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:20:07,852::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:20:07,929::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0331373 s, 30.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:20:08,523::task::595::TaskManager.Task::(_updateState) Task=`eef4f7d3-0ea4-4d00-b04e-8b8f00a2bb6f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:20:08,523::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:20:08,523::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000709587', 'lastCheck': '3.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:20:08,523::task::1185::TaskManager.Task::(prepare) Task=`eef4f7d3-0ea4-4d00-b04e-8b8f00a2bb6f`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000709587', 'lastCheck': '3.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:20:08,523::task::595::TaskManager.Task::(_updateState) Task=`eef4f7d3-0ea4-4d00-b04e-8b8f00a2bb6f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:20:08,524::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:20:08,524::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:20:08,524::task::990::TaskManager.Task::(_decref) Task=`eef4f7d3-0ea4-4d00-b04e-8b8f00a2bb6f`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:20:09,942::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:20:10,020::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0354048 s, 28.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:20:10,297::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:20:12,033::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:20:12,108::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312308 s, 32.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:20:13,319::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:20:14,121::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:20:14,209::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0429894 s, 23.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:20:14,982::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:20:14,990::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000509806 s, 8.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:20:15,825::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [19876d15] >Thread-13::DEBUG::2014-05-13 11:20:15,826::task::595::TaskManager.Task::(_updateState) Task=`4dbb169a-04d1-433c-8389-39c66d3850a9`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:20:15,826::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:20:15,826::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:20:15,826::task::1185::TaskManager.Task::(prepare) Task=`4dbb169a-04d1-433c-8389-39c66d3850a9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:20:15,827::task::595::TaskManager.Task::(_updateState) Task=`4dbb169a-04d1-433c-8389-39c66d3850a9`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:20:15,827::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:20:15,827::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:20:15,827::task::990::TaskManager.Task::(_decref) Task=`4dbb169a-04d1-433c-8389-39c66d3850a9`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:20:15,834::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [19876d15] >Thread-338::DEBUG::2014-05-13 11:20:15,834::task::595::TaskManager.Task::(_updateState) Task=`270a0ca1-fed3-4790-9110-49a84fa753fb`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:20:15,834::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:20:15,835::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`56057874-c534-4c38-a4b2-ce1746528ba0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:20:15,835::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:20:15,835::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:20:15,835::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`56057874-c534-4c38-a4b2-ce1746528ba0`::Granted request >Thread-338::DEBUG::2014-05-13 11:20:15,836::task::827::TaskManager.Task::(resourceAcquired) Task=`270a0ca1-fed3-4790-9110-49a84fa753fb`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:20:15,836::task::990::TaskManager.Task::(_decref) Task=`270a0ca1-fed3-4790-9110-49a84fa753fb`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:20:15,836::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:20:15,837::task::1185::TaskManager.Task::(prepare) Task=`270a0ca1-fed3-4790-9110-49a84fa753fb`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:20:15,837::task::595::TaskManager.Task::(_updateState) Task=`270a0ca1-fed3-4790-9110-49a84fa753fb`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:20:15,837::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:20:15,837::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:20:15,837::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:20:15,838::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:20:15,838::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:20:15,838::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:20:15,838::task::990::TaskManager.Task::(_decref) Task=`270a0ca1-fed3-4790-9110-49a84fa753fb`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:20:16,222::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:20:16,301::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0351568 s, 29.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:20:16,338::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:20:18,313::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:20:18,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307405 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:20:19,361::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:20:20,401::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:20:20,481::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0375402 s, 27.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:20:22,382::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:20:22,494::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:20:22,575::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0380475 s, 26.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:20:23,656::task::595::TaskManager.Task::(_updateState) Task=`f70cef9e-8af8-4214-a05d-89dc57b0da36`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:20:23,656::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:20:23,656::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000509806', 'lastCheck': '8.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:20:23,656::task::1185::TaskManager.Task::(prepare) Task=`f70cef9e-8af8-4214-a05d-89dc57b0da36`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000509806', 'lastCheck': '8.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:20:23,656::task::595::TaskManager.Task::(_updateState) Task=`f70cef9e-8af8-4214-a05d-89dc57b0da36`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:20:23,657::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:20:23,657::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:20:23,657::task::990::TaskManager.Task::(_decref) Task=`f70cef9e-8af8-4214-a05d-89dc57b0da36`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:20:24,587::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:20:24,664::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0339214 s, 30.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:20:24,997::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:20:25,010::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00448924 s, 912 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:20:25,401::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:20:25,858::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:20:25,859::task::595::TaskManager.Task::(_updateState) Task=`334f2236-5faa-4838-a36e-d67eb905bddf`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:20:25,859::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:20:25,859::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:20:25,859::task::1185::TaskManager.Task::(prepare) Task=`334f2236-5faa-4838-a36e-d67eb905bddf`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:20:25,859::task::595::TaskManager.Task::(_updateState) Task=`334f2236-5faa-4838-a36e-d67eb905bddf`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:20:25,860::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:20:25,860::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:20:25,860::task::990::TaskManager.Task::(_decref) Task=`334f2236-5faa-4838-a36e-d67eb905bddf`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:20:25,865::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:20:25,865::task::595::TaskManager.Task::(_updateState) Task=`296482f3-8e76-4ec1-8e37-46c9fd2599bb`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:20:25,865::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:20:25,866::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`446a659c-3647-4dbc-836c-ea19f3914579`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:20:25,866::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:20:25,866::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:20:25,866::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`446a659c-3647-4dbc-836c-ea19f3914579`::Granted request >Thread-338::DEBUG::2014-05-13 11:20:25,867::task::827::TaskManager.Task::(resourceAcquired) Task=`296482f3-8e76-4ec1-8e37-46c9fd2599bb`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:20:25,867::task::990::TaskManager.Task::(_decref) Task=`296482f3-8e76-4ec1-8e37-46c9fd2599bb`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:20:25,867::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:20:25,867::task::1185::TaskManager.Task::(prepare) Task=`296482f3-8e76-4ec1-8e37-46c9fd2599bb`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:20:25,868::task::595::TaskManager.Task::(_updateState) Task=`296482f3-8e76-4ec1-8e37-46c9fd2599bb`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:20:25,868::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:20:25,868::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:20:25,868::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:20:25,868::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:20:25,868::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:20:25,869::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:20:25,869::task::990::TaskManager.Task::(_decref) Task=`296482f3-8e76-4ec1-8e37-46c9fd2599bb`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:20:26,677::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:20:27,044::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.32462 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:20:28,417::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:20:29,057::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:20:29,136::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0342845 s, 29.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:20:31,149::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:20:31,230::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384893 s, 26.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:20:31,441::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:20:33,243::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:20:33,319::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309223 s, 33.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:20:34,460::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:20:35,016::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:20:35,025::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00169648 s, 2.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:20:35,332::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:20:35,437::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0621247 s, 16.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:20:35,889::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [30b3ccf] >Thread-13::DEBUG::2014-05-13 11:20:35,890::task::595::TaskManager.Task::(_updateState) Task=`f6237f6f-7652-4895-90e7-e8e6ea1daaae`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:20:35,890::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:20:35,890::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:20:35,891::task::1185::TaskManager.Task::(prepare) Task=`f6237f6f-7652-4895-90e7-e8e6ea1daaae`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:20:35,891::task::595::TaskManager.Task::(_updateState) Task=`f6237f6f-7652-4895-90e7-e8e6ea1daaae`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:20:35,891::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:20:35,891::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:20:35,891::task::990::TaskManager.Task::(_decref) Task=`f6237f6f-7652-4895-90e7-e8e6ea1daaae`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:20:35,896::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [30b3ccf] >Thread-338::DEBUG::2014-05-13 11:20:35,897::task::595::TaskManager.Task::(_updateState) Task=`62739f8f-9f93-4b8a-a33a-f51bc1b6c0f9`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:20:35,897::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:20:35,897::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`76f57964-27b6-4144-a995-3bc856ebbb9d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:20:35,898::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:20:35,898::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:20:35,898::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`76f57964-27b6-4144-a995-3bc856ebbb9d`::Granted request >Thread-338::DEBUG::2014-05-13 11:20:35,898::task::827::TaskManager.Task::(resourceAcquired) Task=`62739f8f-9f93-4b8a-a33a-f51bc1b6c0f9`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:20:35,899::task::990::TaskManager.Task::(_decref) Task=`62739f8f-9f93-4b8a-a33a-f51bc1b6c0f9`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:20:35,899::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:20:35,899::task::1185::TaskManager.Task::(prepare) Task=`62739f8f-9f93-4b8a-a33a-f51bc1b6c0f9`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:20:35,900::task::595::TaskManager.Task::(_updateState) Task=`62739f8f-9f93-4b8a-a33a-f51bc1b6c0f9`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:20:35,900::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:20:35,900::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:20:35,900::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:20:35,900::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:20:35,901::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:20:35,901::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:20:35,901::task::990::TaskManager.Task::(_decref) Task=`62739f8f-9f93-4b8a-a33a-f51bc1b6c0f9`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:20:37,450::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:20:37,480::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:20:37,532::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387884 s, 26.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:20:38,789::task::595::TaskManager.Task::(_updateState) Task=`df5bce71-3566-497c-851d-37509f5256ea`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:20:38,789::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:20:38,790::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00169648', 'lastCheck': '3.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:20:38,790::task::1185::TaskManager.Task::(prepare) Task=`df5bce71-3566-497c-851d-37509f5256ea`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00169648', 'lastCheck': '3.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:20:38,790::task::595::TaskManager.Task::(_updateState) Task=`df5bce71-3566-497c-851d-37509f5256ea`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:20:38,790::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:20:38,790::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:20:38,790::task::990::TaskManager.Task::(_decref) Task=`df5bce71-3566-497c-851d-37509f5256ea`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:20:39,544::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:20:39,621::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311933 s, 32.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:20:40,510::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:20:41,634::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:20:42,003::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.327162 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:20:43,534::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:20:44,016::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:20:44,090::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0304918 s, 33.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:20:45,033::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:20:45,041::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000581583 s, 7.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:20:45,926::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:20:45,927::task::595::TaskManager.Task::(_updateState) Task=`5a56f158-f177-466a-9999-fd39e1e8ec3c`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:20:45,927::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:20:45,928::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:20:45,928::task::1185::TaskManager.Task::(prepare) Task=`5a56f158-f177-466a-9999-fd39e1e8ec3c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:20:45,928::task::595::TaskManager.Task::(_updateState) Task=`5a56f158-f177-466a-9999-fd39e1e8ec3c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:20:45,928::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:20:45,928::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:20:45,929::task::990::TaskManager.Task::(_decref) Task=`5a56f158-f177-466a-9999-fd39e1e8ec3c`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:20:45,934::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:20:45,934::task::595::TaskManager.Task::(_updateState) Task=`33d2d558-db16-4c2e-80ae-3ca7902437a6`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:20:45,934::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:20:45,935::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`10c594ba-3109-4fe3-a673-fef4bfead4bc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:20:45,935::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:20:45,935::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:20:45,935::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`10c594ba-3109-4fe3-a673-fef4bfead4bc`::Granted request >Thread-338::DEBUG::2014-05-13 11:20:45,936::task::827::TaskManager.Task::(resourceAcquired) Task=`33d2d558-db16-4c2e-80ae-3ca7902437a6`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:20:45,936::task::990::TaskManager.Task::(_decref) Task=`33d2d558-db16-4c2e-80ae-3ca7902437a6`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:20:45,936::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:20:45,937::task::1185::TaskManager.Task::(prepare) Task=`33d2d558-db16-4c2e-80ae-3ca7902437a6`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:20:45,937::task::595::TaskManager.Task::(_updateState) Task=`33d2d558-db16-4c2e-80ae-3ca7902437a6`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:20:45,937::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:20:45,937::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:20:45,937::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:20:45,938::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:20:45,938::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:20:45,938::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:20:45,938::task::990::TaskManager.Task::(_decref) Task=`33d2d558-db16-4c2e-80ae-3ca7902437a6`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:20:46,103::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:20:46,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352591 s, 29.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:20:46,558::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:20:48,195::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:20:48,269::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311311 s, 32.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:20:49,581::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:20:50,282::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:20:50,355::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311222 s, 32.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:20:52,368::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:20:52,462::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0503276 s, 20.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:20:52,603::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:20:53,928::task::595::TaskManager.Task::(_updateState) Task=`abfbefe3-0093-4469-8f96-d97921d6aba1`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:20:53,928::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:20:53,928::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000581583', 'lastCheck': '8.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:20:53,928::task::1185::TaskManager.Task::(prepare) Task=`abfbefe3-0093-4469-8f96-d97921d6aba1`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000581583', 'lastCheck': '8.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:20:53,928::task::595::TaskManager.Task::(_updateState) Task=`abfbefe3-0093-4469-8f96-d97921d6aba1`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:20:53,929::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:20:53,929::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:20:53,929::task::990::TaskManager.Task::(_decref) Task=`abfbefe3-0093-4469-8f96-d97921d6aba1`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:20:54,475::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:20:54,551::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0308356 s, 33.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:20:55,048::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:20:55,057::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000557209 s, 7.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:20:55,623::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:20:55,963::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4478bd69] >Thread-13::DEBUG::2014-05-13 11:20:55,963::task::595::TaskManager.Task::(_updateState) Task=`1ff21290-6660-470c-b7ad-bd4e2aa7ba9c`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:20:55,963::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:20:55,964::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:20:55,964::task::1185::TaskManager.Task::(prepare) Task=`1ff21290-6660-470c-b7ad-bd4e2aa7ba9c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:20:55,964::task::595::TaskManager.Task::(_updateState) Task=`1ff21290-6660-470c-b7ad-bd4e2aa7ba9c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:20:55,964::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:20:55,964::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:20:55,965::task::990::TaskManager.Task::(_decref) Task=`1ff21290-6660-470c-b7ad-bd4e2aa7ba9c`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:20:55,973::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4478bd69] >Thread-338::DEBUG::2014-05-13 11:20:55,973::task::595::TaskManager.Task::(_updateState) Task=`560c6a87-bd77-43cf-a4e7-3815644ecf6c`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:20:55,973::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:20:55,974::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`134f4532-c0b6-4683-992d-b83f36858f17`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:20:55,974::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:20:55,974::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:20:55,974::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`134f4532-c0b6-4683-992d-b83f36858f17`::Granted request >Thread-338::DEBUG::2014-05-13 11:20:55,975::task::827::TaskManager.Task::(resourceAcquired) Task=`560c6a87-bd77-43cf-a4e7-3815644ecf6c`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:20:55,975::task::990::TaskManager.Task::(_decref) Task=`560c6a87-bd77-43cf-a4e7-3815644ecf6c`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:20:55,975::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:20:55,976::task::1185::TaskManager.Task::(prepare) Task=`560c6a87-bd77-43cf-a4e7-3815644ecf6c`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:20:55,976::task::595::TaskManager.Task::(_updateState) Task=`560c6a87-bd77-43cf-a4e7-3815644ecf6c`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:20:55,976::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:20:55,976::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:20:55,976::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:20:55,976::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:20:55,977::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:20:55,977::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:20:55,977::task::990::TaskManager.Task::(_decref) Task=`560c6a87-bd77-43cf-a4e7-3815644ecf6c`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:20:56,564::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:20:56,637::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0301474 s, 34.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:20:58,642::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:20:58,650::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:20:58,724::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.030762 s, 33.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:21:00,738::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:00,819::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381866 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:21:01,660::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:21:02,832::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:02,906::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307775 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:21:04,679::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:21:04,919::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:04,993::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0308765 s, 33.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:21:05,066::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:21:05,075::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000755353 s, 5.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:21:05,996::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17998964] >Thread-13::DEBUG::2014-05-13 11:21:06,019::task::595::TaskManager.Task::(_updateState) Task=`503c043d-e4cb-4f10-8428-db7fd85fc633`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:21:06,019::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:21:06,019::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:21:06,020::task::1185::TaskManager.Task::(prepare) Task=`503c043d-e4cb-4f10-8428-db7fd85fc633`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:21:06,020::task::595::TaskManager.Task::(_updateState) Task=`503c043d-e4cb-4f10-8428-db7fd85fc633`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:21:06,020::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:21:06,020::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:21:06,020::task::990::TaskManager.Task::(_decref) Task=`503c043d-e4cb-4f10-8428-db7fd85fc633`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:21:06,025::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17998964] >Thread-338::DEBUG::2014-05-13 11:21:06,025::task::595::TaskManager.Task::(_updateState) Task=`3280a42b-3f4d-4751-a13e-be0b08368017`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:21:06,026::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:21:06,026::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`61f2c0c5-b715-41b3-8a72-35f69238ffea`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:21:06,026::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:21:06,027::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:21:06,027::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`61f2c0c5-b715-41b3-8a72-35f69238ffea`::Granted request >Thread-338::DEBUG::2014-05-13 11:21:06,027::task::827::TaskManager.Task::(resourceAcquired) Task=`3280a42b-3f4d-4751-a13e-be0b08368017`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:21:06,027::task::990::TaskManager.Task::(_decref) Task=`3280a42b-3f4d-4751-a13e-be0b08368017`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:21:06,028::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:21:06,028::task::1185::TaskManager.Task::(prepare) Task=`3280a42b-3f4d-4751-a13e-be0b08368017`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:21:06,028::task::595::TaskManager.Task::(_updateState) Task=`3280a42b-3f4d-4751-a13e-be0b08368017`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:21:06,029::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:21:06,029::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:21:06,029::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:21:06,029::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:21:06,029::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:21:06,029::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:21:06,030::task::990::TaskManager.Task::(_decref) Task=`3280a42b-3f4d-4751-a13e-be0b08368017`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:21:07,006::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:07,100::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0514192 s, 19.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:21:07,704::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:21:09,107::task::595::TaskManager.Task::(_updateState) Task=`953a884b-47f1-44c6-a1dd-c11246bf7150`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:21:09,108::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:21:09,108::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000755353', 'lastCheck': '4.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:21:09,108::task::1185::TaskManager.Task::(prepare) Task=`953a884b-47f1-44c6-a1dd-c11246bf7150`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000755353', 'lastCheck': '4.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:21:09,108::task::595::TaskManager.Task::(_updateState) Task=`953a884b-47f1-44c6-a1dd-c11246bf7150`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:21:09,108::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:21:09,108::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:21:09,109::task::990::TaskManager.Task::(_decref) Task=`953a884b-47f1-44c6-a1dd-c11246bf7150`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:21:09,113::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:09,187::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306361 s, 33.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:21:10,729::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:21:11,200::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:11,274::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314373 s, 32.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:21:13,287::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:13,367::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360448 s, 28.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:21:13,750::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:21:15,082::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:21:15,090::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000787713 s, 5.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:21:15,380::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:15,459::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358734 s, 28.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:21:16,061::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:21:16,062::task::595::TaskManager.Task::(_updateState) Task=`64855105-a22b-4a93-a0aa-a04fb2322eb2`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:21:16,062::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:21:16,063::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:21:16,063::task::1185::TaskManager.Task::(prepare) Task=`64855105-a22b-4a93-a0aa-a04fb2322eb2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:21:16,063::task::595::TaskManager.Task::(_updateState) Task=`64855105-a22b-4a93-a0aa-a04fb2322eb2`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:21:16,063::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:21:16,063::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:21:16,063::task::990::TaskManager.Task::(_decref) Task=`64855105-a22b-4a93-a0aa-a04fb2322eb2`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:21:16,068::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:21:16,068::task::595::TaskManager.Task::(_updateState) Task=`3682fe27-f451-4f5a-a7a9-a26be6122935`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:21:16,068::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:21:16,069::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`17fdb5ad-c2fd-4f12-8707-bbc3e996962e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:21:16,069::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:21:16,069::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:21:16,069::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`17fdb5ad-c2fd-4f12-8707-bbc3e996962e`::Granted request >Thread-338::DEBUG::2014-05-13 11:21:16,070::task::827::TaskManager.Task::(resourceAcquired) Task=`3682fe27-f451-4f5a-a7a9-a26be6122935`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:21:16,070::task::990::TaskManager.Task::(_decref) Task=`3682fe27-f451-4f5a-a7a9-a26be6122935`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:21:16,071::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:21:16,071::task::1185::TaskManager.Task::(prepare) Task=`3682fe27-f451-4f5a-a7a9-a26be6122935`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:21:16,071::task::595::TaskManager.Task::(_updateState) Task=`3682fe27-f451-4f5a-a7a9-a26be6122935`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:21:16,071::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:21:16,071::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:21:16,072::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:21:16,072::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:21:16,072::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:21:16,072::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:21:16,072::task::990::TaskManager.Task::(_decref) Task=`3682fe27-f451-4f5a-a7a9-a26be6122935`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:21:16,770::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:21:17,488::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:17,561::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0304495 s, 33.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:21:19,574::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:19,651::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0371244 s, 27.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:21:19,789::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:21:21,659::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:21,751::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0501182 s, 20.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:21:22,807::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:21:23,764::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:23,838::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303831 s, 33.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:21:24,238::task::595::TaskManager.Task::(_updateState) Task=`296d90de-6734-4f62-8b71-4163e440b3de`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:21:24,238::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:21:24,239::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000787713', 'lastCheck': '9.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:21:24,239::task::1185::TaskManager.Task::(prepare) Task=`296d90de-6734-4f62-8b71-4163e440b3de`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000787713', 'lastCheck': '9.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:21:24,239::task::595::TaskManager.Task::(_updateState) Task=`296d90de-6734-4f62-8b71-4163e440b3de`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:21:24,239::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:21:24,239::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:21:24,240::task::990::TaskManager.Task::(_decref) Task=`296d90de-6734-4f62-8b71-4163e440b3de`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 11:21:25,098::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:21:25,106::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000584515 s, 7.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:21:25,825::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:21:25,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:25,925::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322639 s, 31.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:21:26,090::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [dc1f215] >Thread-13::DEBUG::2014-05-13 11:21:26,091::task::595::TaskManager.Task::(_updateState) Task=`4686ee10-5b68-40d4-89e3-8fda7e4092d9`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:21:26,091::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:21:26,091::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:21:26,091::task::1185::TaskManager.Task::(prepare) Task=`4686ee10-5b68-40d4-89e3-8fda7e4092d9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:21:26,092::task::595::TaskManager.Task::(_updateState) Task=`4686ee10-5b68-40d4-89e3-8fda7e4092d9`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:21:26,092::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:21:26,092::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:21:26,092::task::990::TaskManager.Task::(_decref) Task=`4686ee10-5b68-40d4-89e3-8fda7e4092d9`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:21:26,097::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [dc1f215] >Thread-338::DEBUG::2014-05-13 11:21:26,097::task::595::TaskManager.Task::(_updateState) Task=`b9f21c6f-6d2e-4f56-9697-83ea8be408db`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:21:26,097::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:21:26,098::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b0b3e0b6-bc5b-4805-8e15-21dc25310b15`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:21:26,098::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:21:26,098::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:21:26,098::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b0b3e0b6-bc5b-4805-8e15-21dc25310b15`::Granted request >Thread-338::DEBUG::2014-05-13 11:21:26,099::task::827::TaskManager.Task::(resourceAcquired) Task=`b9f21c6f-6d2e-4f56-9697-83ea8be408db`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:21:26,100::task::990::TaskManager.Task::(_decref) Task=`b9f21c6f-6d2e-4f56-9697-83ea8be408db`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:21:26,100::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:21:26,101::task::1185::TaskManager.Task::(prepare) Task=`b9f21c6f-6d2e-4f56-9697-83ea8be408db`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:21:26,101::task::595::TaskManager.Task::(_updateState) Task=`b9f21c6f-6d2e-4f56-9697-83ea8be408db`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:21:26,101::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:21:26,101::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:21:26,101::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:21:26,101::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:21:26,102::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:21:26,102::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:21:26,102::task::990::TaskManager.Task::(_decref) Task=`b9f21c6f-6d2e-4f56-9697-83ea8be408db`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:21:27,939::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:28,017::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0349229 s, 29.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:21:28,846::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:21:30,030::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:30,105::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.032409 s, 31.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:21:31,867::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:21:32,118::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:32,191::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0299146 s, 34.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:21:34,204::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:34,303::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0570917 s, 17.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:21:34,885::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:21:35,113::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:21:35,121::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000643434 s, 6.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:21:36,120::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3c71a9dd] >Thread-13::DEBUG::2014-05-13 11:21:36,121::task::595::TaskManager.Task::(_updateState) Task=`5bab7d0a-661b-454c-a087-d523c74989cc`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:21:36,121::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:21:36,121::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:21:36,122::task::1185::TaskManager.Task::(prepare) Task=`5bab7d0a-661b-454c-a087-d523c74989cc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:21:36,122::task::595::TaskManager.Task::(_updateState) Task=`5bab7d0a-661b-454c-a087-d523c74989cc`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:21:36,122::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:21:36,122::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:21:36,122::task::990::TaskManager.Task::(_decref) Task=`5bab7d0a-661b-454c-a087-d523c74989cc`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:21:36,127::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3c71a9dd] >Thread-338::DEBUG::2014-05-13 11:21:36,127::task::595::TaskManager.Task::(_updateState) Task=`3a33ba52-8dbd-4b88-b460-082bbd0be6a9`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:21:36,127::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:21:36,128::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b5761a05-7547-43c9-a991-d2372fefa1f3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:21:36,128::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:21:36,128::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:21:36,128::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b5761a05-7547-43c9-a991-d2372fefa1f3`::Granted request >Thread-338::DEBUG::2014-05-13 11:21:36,129::task::827::TaskManager.Task::(resourceAcquired) Task=`3a33ba52-8dbd-4b88-b460-082bbd0be6a9`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:21:36,129::task::990::TaskManager.Task::(_decref) Task=`3a33ba52-8dbd-4b88-b460-082bbd0be6a9`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:21:36,130::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:21:36,130::task::1185::TaskManager.Task::(prepare) Task=`3a33ba52-8dbd-4b88-b460-082bbd0be6a9`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:21:36,130::task::595::TaskManager.Task::(_updateState) Task=`3a33ba52-8dbd-4b88-b460-082bbd0be6a9`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:21:36,130::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:21:36,130::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:21:36,131::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:21:36,131::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:21:36,131::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:21:36,131::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:21:36,131::task::990::TaskManager.Task::(_decref) Task=`3a33ba52-8dbd-4b88-b460-082bbd0be6a9`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:21:36,316::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:36,403::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0450796 s, 22.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:21:37,903::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:21:38,417::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:38,492::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0330274 s, 31.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:21:39,377::task::595::TaskManager.Task::(_updateState) Task=`b71285b9-5b97-4ecb-908b-930a1727f94e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:21:39,378::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:21:39,378::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000643434', 'lastCheck': '4.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:21:39,378::task::1185::TaskManager.Task::(prepare) Task=`b71285b9-5b97-4ecb-908b-930a1727f94e`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000643434', 'lastCheck': '4.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:21:39,378::task::595::TaskManager.Task::(_updateState) Task=`b71285b9-5b97-4ecb-908b-930a1727f94e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:21:39,378::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:21:39,379::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:21:39,379::task::990::TaskManager.Task::(_decref) Task=`b71285b9-5b97-4ecb-908b-930a1727f94e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:21:40,505::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:40,584::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0370586 s, 27.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:21:40,922::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:21:42,597::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:42,679::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0337567 s, 30.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:21:43,946::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:21:44,693::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:44,771::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0335668 s, 30.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:21:45,128::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:21:45,137::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00073267 s, 5.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:21:46,150::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [50b954f9] >Thread-13::DEBUG::2014-05-13 11:21:46,150::task::595::TaskManager.Task::(_updateState) Task=`e315fe46-2b65-4784-8057-cf960af6eeda`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:21:46,151::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:21:46,151::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:21:46,151::task::1185::TaskManager.Task::(prepare) Task=`e315fe46-2b65-4784-8057-cf960af6eeda`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:21:46,151::task::595::TaskManager.Task::(_updateState) Task=`e315fe46-2b65-4784-8057-cf960af6eeda`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:21:46,152::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:21:46,152::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:21:46,152::task::990::TaskManager.Task::(_decref) Task=`e315fe46-2b65-4784-8057-cf960af6eeda`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:21:46,156::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [50b954f9] >Thread-338::DEBUG::2014-05-13 11:21:46,157::task::595::TaskManager.Task::(_updateState) Task=`e4bcdcd9-878c-4783-9a93-d8914709eeb3`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:21:46,157::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:21:46,157::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0a4082fd-66a4-4051-93ef-ed98c1318250`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:21:46,158::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:21:46,158::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:21:46,158::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0a4082fd-66a4-4051-93ef-ed98c1318250`::Granted request >Thread-338::DEBUG::2014-05-13 11:21:46,158::task::827::TaskManager.Task::(resourceAcquired) Task=`e4bcdcd9-878c-4783-9a93-d8914709eeb3`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:21:46,159::task::990::TaskManager.Task::(_decref) Task=`e4bcdcd9-878c-4783-9a93-d8914709eeb3`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:21:46,159::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:21:46,160::task::1185::TaskManager.Task::(prepare) Task=`e4bcdcd9-878c-4783-9a93-d8914709eeb3`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:21:46,160::task::595::TaskManager.Task::(_updateState) Task=`e4bcdcd9-878c-4783-9a93-d8914709eeb3`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:21:46,160::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:21:46,160::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:21:46,160::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:21:46,161::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:21:46,161::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:21:46,161::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:21:46,161::task::990::TaskManager.Task::(_decref) Task=`e4bcdcd9-878c-4783-9a93-d8914709eeb3`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:21:46,785::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:46,877::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0428663 s, 23.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:21:46,973::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:21:48,890::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:48,983::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0486451 s, 21.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:21:49,992::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:21:50,997::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:51,076::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0369283 s, 27.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:21:53,021::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:21:53,090::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:53,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0344053 s, 29.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:21:54,522::task::595::TaskManager.Task::(_updateState) Task=`0676696d-1ecb-4e3b-9227-bb160c3bf224`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:21:54,523::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:21:54,523::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00073267', 'lastCheck': '9.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:21:54,523::task::1185::TaskManager.Task::(prepare) Task=`0676696d-1ecb-4e3b-9227-bb160c3bf224`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00073267', 'lastCheck': '9.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:21:54,523::task::595::TaskManager.Task::(_updateState) Task=`0676696d-1ecb-4e3b-9227-bb160c3bf224`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:21:54,524::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:21:54,524::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:21:54,524::task::990::TaskManager.Task::(_decref) Task=`0676696d-1ecb-4e3b-9227-bb160c3bf224`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 11:21:55,145::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:21:55,153::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000619472 s, 6.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:21:55,180::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:55,258::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355992 s, 28.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:21:56,042::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:21:56,181::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:21:56,182::task::595::TaskManager.Task::(_updateState) Task=`db7ed99c-be26-4e9a-b207-d313006071a9`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:21:56,182::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:21:56,182::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:21:56,182::task::1185::TaskManager.Task::(prepare) Task=`db7ed99c-be26-4e9a-b207-d313006071a9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:21:56,182::task::595::TaskManager.Task::(_updateState) Task=`db7ed99c-be26-4e9a-b207-d313006071a9`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:21:56,183::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:21:56,183::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:21:56,183::task::990::TaskManager.Task::(_decref) Task=`db7ed99c-be26-4e9a-b207-d313006071a9`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:21:56,188::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:21:56,188::task::595::TaskManager.Task::(_updateState) Task=`28e98c0a-1801-4a7a-825c-08900e00108e`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:21:56,188::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:21:56,188::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d19c83bb-2f86-4a79-88e8-6ee0c41830b8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:21:56,189::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:21:56,189::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:21:56,189::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d19c83bb-2f86-4a79-88e8-6ee0c41830b8`::Granted request >Thread-338::DEBUG::2014-05-13 11:21:56,189::task::827::TaskManager.Task::(resourceAcquired) Task=`28e98c0a-1801-4a7a-825c-08900e00108e`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:21:56,190::task::990::TaskManager.Task::(_decref) Task=`28e98c0a-1801-4a7a-825c-08900e00108e`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:21:56,190::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:21:56,191::task::1185::TaskManager.Task::(prepare) Task=`28e98c0a-1801-4a7a-825c-08900e00108e`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:21:56,191::task::595::TaskManager.Task::(_updateState) Task=`28e98c0a-1801-4a7a-825c-08900e00108e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:21:56,191::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:21:56,191::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:21:56,191::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:21:56,192::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:21:56,192::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:21:56,192::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:21:56,192::task::990::TaskManager.Task::(_decref) Task=`28e98c0a-1801-4a7a-825c-08900e00108e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:21:57,271::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:57,351::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0367005 s, 27.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:21:59,060::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:21:59,365::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:21:59,446::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313505 s, 32.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:22:01,459::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:01,540::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365329 s, 28.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:22:02,079::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:22:03,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:03,661::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0611899 s, 16.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:22:05,097::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:22:05,160::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:22:05,168::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000531393 s, 7.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:22:05,674::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:05,751::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0348481 s, 29.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:22:06,211::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:22:06,212::task::595::TaskManager.Task::(_updateState) Task=`8ed7cd39-75aa-4e62-a85f-c72139d5533c`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:22:06,212::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:22:06,212::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:22:06,212::task::1185::TaskManager.Task::(prepare) Task=`8ed7cd39-75aa-4e62-a85f-c72139d5533c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:22:06,213::task::595::TaskManager.Task::(_updateState) Task=`8ed7cd39-75aa-4e62-a85f-c72139d5533c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:22:06,213::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:22:06,213::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:22:06,213::task::990::TaskManager.Task::(_decref) Task=`8ed7cd39-75aa-4e62-a85f-c72139d5533c`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:22:06,219::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:22:06,219::task::595::TaskManager.Task::(_updateState) Task=`e2964896-4f03-4dfe-9241-1500337a7d12`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:22:06,219::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:22:06,220::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`cc534939-e7fe-4302-b7ed-cbfa3c0db227`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:22:06,220::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:22:06,220::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:22:06,220::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`cc534939-e7fe-4302-b7ed-cbfa3c0db227`::Granted request >Thread-338::DEBUG::2014-05-13 11:22:06,221::task::827::TaskManager.Task::(resourceAcquired) Task=`e2964896-4f03-4dfe-9241-1500337a7d12`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:22:06,221::task::990::TaskManager.Task::(_decref) Task=`e2964896-4f03-4dfe-9241-1500337a7d12`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:22:06,222::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:22:06,222::task::1185::TaskManager.Task::(prepare) Task=`e2964896-4f03-4dfe-9241-1500337a7d12`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:22:06,222::task::595::TaskManager.Task::(_updateState) Task=`e2964896-4f03-4dfe-9241-1500337a7d12`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:22:06,222::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:22:06,222::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:22:06,223::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:22:06,223::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:22:06,223::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:22:06,223::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:22:06,223::task::990::TaskManager.Task::(_decref) Task=`e2964896-4f03-4dfe-9241-1500337a7d12`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:22:07,764::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:07,838::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307085 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:22:08,123::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:22:09,660::task::595::TaskManager.Task::(_updateState) Task=`8ac6871c-6c94-412c-8a85-d51653ea611e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:22:09,661::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:22:09,661::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000531393', 'lastCheck': '4.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:22:09,661::task::1185::TaskManager.Task::(prepare) Task=`8ac6871c-6c94-412c-8a85-d51653ea611e`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000531393', 'lastCheck': '4.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:22:09,661::task::595::TaskManager.Task::(_updateState) Task=`8ac6871c-6c94-412c-8a85-d51653ea611e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:22:09,661::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:22:09,661::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:22:09,662::task::990::TaskManager.Task::(_decref) Task=`8ac6871c-6c94-412c-8a85-d51653ea611e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:22:09,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:10,089::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353126 s, 29.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:22:11,142::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:22:12,106::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:12,186::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.03689 s, 27.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:22:14,162::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:22:14,199::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:14,273::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0304735 s, 33.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:22:15,175::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:22:15,184::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000666214 s, 6.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:22:16,243::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4a63f6c8] >Thread-13::DEBUG::2014-05-13 11:22:16,268::task::595::TaskManager.Task::(_updateState) Task=`3bd8b11b-7fd5-40c3-ba90-a912dec51256`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:22:16,268::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:22:16,268::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:22:16,269::task::1185::TaskManager.Task::(prepare) Task=`3bd8b11b-7fd5-40c3-ba90-a912dec51256`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:22:16,269::task::595::TaskManager.Task::(_updateState) Task=`3bd8b11b-7fd5-40c3-ba90-a912dec51256`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:22:16,269::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:22:16,269::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:22:16,269::task::990::TaskManager.Task::(_decref) Task=`3bd8b11b-7fd5-40c3-ba90-a912dec51256`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:22:16,274::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4a63f6c8] >Thread-338::DEBUG::2014-05-13 11:22:16,275::task::595::TaskManager.Task::(_updateState) Task=`c4f582f8-252e-4b52-82f5-14ebe8fbf18a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:22:16,275::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:22:16,275::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2d2170b6-c4c9-4cdd-baac-fd39aa793044`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:22:16,276::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:22:16,276::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:22:16,276::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2d2170b6-c4c9-4cdd-baac-fd39aa793044`::Granted request >Thread-338::DEBUG::2014-05-13 11:22:16,276::task::827::TaskManager.Task::(resourceAcquired) Task=`c4f582f8-252e-4b52-82f5-14ebe8fbf18a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:22:16,276::task::990::TaskManager.Task::(_decref) Task=`c4f582f8-252e-4b52-82f5-14ebe8fbf18a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:22:16,277::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:22:16,277::task::1185::TaskManager.Task::(prepare) Task=`c4f582f8-252e-4b52-82f5-14ebe8fbf18a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:22:16,278::task::595::TaskManager.Task::(_updateState) Task=`c4f582f8-252e-4b52-82f5-14ebe8fbf18a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:22:16,278::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:22:16,278::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:22:16,278::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:22:16,278::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:22:16,279::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:22:16,279::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:22:16,279::task::990::TaskManager.Task::(_decref) Task=`c4f582f8-252e-4b52-82f5-14ebe8fbf18a`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:22:16,286::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:16,699::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.369982 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:22:17,184::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:22:18,708::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:18,789::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.033043 s, 31.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:22:20,203::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:22:20,801::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:20,881::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0340418 s, 30.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:22:22,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:22,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0296473 s, 34.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:22:23,223::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:22:24,797::task::595::TaskManager.Task::(_updateState) Task=`e244551d-fa0c-4c15-9ae6-4a3293a5b6b4`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:22:24,797::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:22:24,797::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000666214', 'lastCheck': '9.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:22:24,798::task::1185::TaskManager.Task::(prepare) Task=`e244551d-fa0c-4c15-9ae6-4a3293a5b6b4`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000666214', 'lastCheck': '9.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:22:24,798::task::595::TaskManager.Task::(_updateState) Task=`e244551d-fa0c-4c15-9ae6-4a3293a5b6b4`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:22:24,798::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:22:24,798::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:22:24,798::task::990::TaskManager.Task::(_decref) Task=`e244551d-fa0c-4c15-9ae6-4a3293a5b6b4`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:22:24,980::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:25,059::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0348817 s, 29.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:22:25,192::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:22:25,200::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000662675 s, 6.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:22:26,242::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:22:26,297::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:22:26,298::task::595::TaskManager.Task::(_updateState) Task=`dd25343d-0491-4343-bde6-123917ad9578`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:22:26,298::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:22:26,298::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:22:26,299::task::1185::TaskManager.Task::(prepare) Task=`dd25343d-0491-4343-bde6-123917ad9578`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:22:26,299::task::595::TaskManager.Task::(_updateState) Task=`dd25343d-0491-4343-bde6-123917ad9578`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:22:26,299::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:22:26,299::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:22:26,299::task::990::TaskManager.Task::(_decref) Task=`dd25343d-0491-4343-bde6-123917ad9578`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:22:26,305::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:22:26,305::task::595::TaskManager.Task::(_updateState) Task=`49f552fa-db50-4c3d-ba86-17c77c608d38`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:22:26,305::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:22:26,306::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`cd5b9827-c905-4e0c-bf41-d2d6f9f047cd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:22:26,306::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:22:26,306::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:22:26,307::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`cd5b9827-c905-4e0c-bf41-d2d6f9f047cd`::Granted request >Thread-338::DEBUG::2014-05-13 11:22:26,307::task::827::TaskManager.Task::(resourceAcquired) Task=`49f552fa-db50-4c3d-ba86-17c77c608d38`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:22:26,307::task::990::TaskManager.Task::(_decref) Task=`49f552fa-db50-4c3d-ba86-17c77c608d38`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:22:26,308::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:22:26,308::task::1185::TaskManager.Task::(prepare) Task=`49f552fa-db50-4c3d-ba86-17c77c608d38`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:22:26,308::task::595::TaskManager.Task::(_updateState) Task=`49f552fa-db50-4c3d-ba86-17c77c608d38`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:22:26,308::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:22:26,309::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:22:26,309::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:22:26,309::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:22:26,309::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:22:26,309::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:22:26,310::task::990::TaskManager.Task::(_decref) Task=`49f552fa-db50-4c3d-ba86-17c77c608d38`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:22:27,072::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:27,151::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355893 s, 28.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:22:29,164::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:29,251::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.043541 s, 23.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:22:29,260::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:22:31,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:31,346::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0354511 s, 28.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:22:32,278::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:22:33,358::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:33,434::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322736 s, 31.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:22:35,207::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:22:35,215::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000722824 s, 5.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:22:35,298::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:22:35,447::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:35,526::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361938 s, 28.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:22:36,329::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6623fb10] >Thread-13::DEBUG::2014-05-13 11:22:36,330::task::595::TaskManager.Task::(_updateState) Task=`f56df047-2322-4439-9591-1d6d6360b427`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:22:36,330::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:22:36,331::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:22:36,331::task::1185::TaskManager.Task::(prepare) Task=`f56df047-2322-4439-9591-1d6d6360b427`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:22:36,331::task::595::TaskManager.Task::(_updateState) Task=`f56df047-2322-4439-9591-1d6d6360b427`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:22:36,331::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:22:36,331::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:22:36,332::task::990::TaskManager.Task::(_decref) Task=`f56df047-2322-4439-9591-1d6d6360b427`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:22:36,638::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6623fb10] >Thread-338::DEBUG::2014-05-13 11:22:36,639::task::595::TaskManager.Task::(_updateState) Task=`b6380a43-2c52-493e-8663-1f4a32d9c0e8`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:22:36,639::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:22:36,639::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a21cf493-cd38-41c9-800e-a21b817c739e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:22:36,639::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:22:36,640::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:22:36,640::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a21cf493-cd38-41c9-800e-a21b817c739e`::Granted request >Thread-338::DEBUG::2014-05-13 11:22:36,641::task::827::TaskManager.Task::(resourceAcquired) Task=`b6380a43-2c52-493e-8663-1f4a32d9c0e8`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:22:36,641::task::990::TaskManager.Task::(_decref) Task=`b6380a43-2c52-493e-8663-1f4a32d9c0e8`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:22:36,642::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:22:36,642::task::1185::TaskManager.Task::(prepare) Task=`b6380a43-2c52-493e-8663-1f4a32d9c0e8`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:22:36,643::task::595::TaskManager.Task::(_updateState) Task=`b6380a43-2c52-493e-8663-1f4a32d9c0e8`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:22:36,643::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:22:36,643::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:22:36,643::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:22:36,643::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:22:36,643::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:22:36,644::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:22:36,644::task::990::TaskManager.Task::(_decref) Task=`b6380a43-2c52-493e-8663-1f4a32d9c0e8`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:22:37,539::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:37,617::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0343954 s, 29.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:22:38,317::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:22:39,630::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:39,724::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305531 s, 33.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:22:39,937::task::595::TaskManager.Task::(_updateState) Task=`91993726-b02b-42c2-afb9-857d5072541d`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:22:39,937::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:22:39,937::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000722824', 'lastCheck': '4.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:22:39,937::task::1185::TaskManager.Task::(prepare) Task=`91993726-b02b-42c2-afb9-857d5072541d`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000722824', 'lastCheck': '4.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:22:39,938::task::595::TaskManager.Task::(_updateState) Task=`91993726-b02b-42c2-afb9-857d5072541d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:22:39,938::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:22:39,938::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:22:39,938::task::990::TaskManager.Task::(_decref) Task=`91993726-b02b-42c2-afb9-857d5072541d`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:22:41,336::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:22:41,737::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:41,816::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0366264 s, 28.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:22:43,829::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:43,916::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.043781 s, 23.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:22:44,353::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:22:45,223::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:22:45,231::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000668853 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:22:45,930::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:46,011::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383878 s, 26.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:22:46,673::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:22:46,674::task::595::TaskManager.Task::(_updateState) Task=`d1c0edb3-14e6-43c3-9c4a-17ad0fe712b4`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:22:46,674::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:22:46,674::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:22:46,675::task::1185::TaskManager.Task::(prepare) Task=`d1c0edb3-14e6-43c3-9c4a-17ad0fe712b4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:22:46,675::task::595::TaskManager.Task::(_updateState) Task=`d1c0edb3-14e6-43c3-9c4a-17ad0fe712b4`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:22:46,675::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:22:46,675::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:22:46,675::task::990::TaskManager.Task::(_decref) Task=`d1c0edb3-14e6-43c3-9c4a-17ad0fe712b4`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:22:46,681::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:22:46,681::task::595::TaskManager.Task::(_updateState) Task=`95af5d9f-3fe3-45c6-a524-f81811ef5900`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:22:46,681::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:22:46,682::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`34935435-da36-47e1-b127-b657e366e1fe`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:22:46,682::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:22:46,682::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:22:46,682::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`34935435-da36-47e1-b127-b657e366e1fe`::Granted request >Thread-338::DEBUG::2014-05-13 11:22:46,683::task::827::TaskManager.Task::(resourceAcquired) Task=`95af5d9f-3fe3-45c6-a524-f81811ef5900`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:22:46,683::task::990::TaskManager.Task::(_decref) Task=`95af5d9f-3fe3-45c6-a524-f81811ef5900`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:22:46,684::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:22:46,684::task::1185::TaskManager.Task::(prepare) Task=`95af5d9f-3fe3-45c6-a524-f81811ef5900`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:22:46,684::task::595::TaskManager.Task::(_updateState) Task=`95af5d9f-3fe3-45c6-a524-f81811ef5900`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:22:46,684::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:22:46,684::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:22:46,684::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:22:46,685::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:22:46,685::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:22:46,685::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:22:46,685::task::990::TaskManager.Task::(_decref) Task=`95af5d9f-3fe3-45c6-a524-f81811ef5900`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:22:47,370::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:22:48,024::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:48,098::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321745 s, 31.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:22:50,111::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:50,186::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031573 s, 32.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:22:50,390::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:22:52,260::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:52,338::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0349196 s, 29.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:22:53,409::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:22:54,351::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:54,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0300256 s, 34.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:22:55,073::task::595::TaskManager.Task::(_updateState) Task=`e32c3015-b026-41b7-a029-d326e43ee35c`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:22:55,074::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:22:55,074::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000668853', 'lastCheck': '9.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:22:55,074::task::1185::TaskManager.Task::(prepare) Task=`e32c3015-b026-41b7-a029-d326e43ee35c`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000668853', 'lastCheck': '9.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:22:55,074::task::595::TaskManager.Task::(_updateState) Task=`e32c3015-b026-41b7-a029-d326e43ee35c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:22:55,074::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:22:55,075::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:22:55,075::task::990::TaskManager.Task::(_decref) Task=`e32c3015-b026-41b7-a029-d326e43ee35c`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 11:22:55,238::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:22:55,246::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000598043 s, 6.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:22:56,428::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:22:56,440::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:56,519::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352622 s, 29.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:22:56,705::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [dc1f215] >Thread-13::DEBUG::2014-05-13 11:22:56,706::task::595::TaskManager.Task::(_updateState) Task=`4fada6f3-b43a-484a-858c-8713bb9af14a`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:22:56,706::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:22:56,707::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:22:56,707::task::1185::TaskManager.Task::(prepare) Task=`4fada6f3-b43a-484a-858c-8713bb9af14a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:22:56,707::task::595::TaskManager.Task::(_updateState) Task=`4fada6f3-b43a-484a-858c-8713bb9af14a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:22:56,707::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:22:56,707::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:22:56,707::task::990::TaskManager.Task::(_decref) Task=`4fada6f3-b43a-484a-858c-8713bb9af14a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:22:56,712::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [dc1f215] >Thread-338::DEBUG::2014-05-13 11:22:56,712::task::595::TaskManager.Task::(_updateState) Task=`982c2d47-20a3-4197-bdf9-60f17786407b`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:22:56,713::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:22:56,713::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e0b6cf18-71e1-46ed-97ea-5c5a96a0a045`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:22:56,713::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:22:56,713::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:22:56,714::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e0b6cf18-71e1-46ed-97ea-5c5a96a0a045`::Granted request >Thread-338::DEBUG::2014-05-13 11:22:56,714::task::827::TaskManager.Task::(resourceAcquired) Task=`982c2d47-20a3-4197-bdf9-60f17786407b`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:22:56,714::task::990::TaskManager.Task::(_decref) Task=`982c2d47-20a3-4197-bdf9-60f17786407b`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:22:56,715::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:22:56,715::task::1185::TaskManager.Task::(prepare) Task=`982c2d47-20a3-4197-bdf9-60f17786407b`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:22:56,715::task::595::TaskManager.Task::(_updateState) Task=`982c2d47-20a3-4197-bdf9-60f17786407b`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:22:56,715::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:22:56,716::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:22:56,716::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:22:56,716::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:22:56,716::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:22:56,716::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:22:56,717::task::990::TaskManager.Task::(_decref) Task=`982c2d47-20a3-4197-bdf9-60f17786407b`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:22:58,532::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:22:58,895::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.320779 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:22:59,456::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:23:00,907::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:23:00,988::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0377872 s, 27.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:23:02,471::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:23:03,001::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:23:03,075::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0302273 s, 33.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:23:05,089::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:23:05,162::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.030005 s, 34.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:23:05,254::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:23:05,262::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000665588 s, 6.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:23:05,494::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:23:06,734::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3c71a9dd] >Thread-13::DEBUG::2014-05-13 11:23:06,735::task::595::TaskManager.Task::(_updateState) Task=`f99a8c06-f1e9-4bd5-88fa-647872bd96f7`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:23:06,735::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:23:06,736::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:23:06,736::task::1185::TaskManager.Task::(prepare) Task=`f99a8c06-f1e9-4bd5-88fa-647872bd96f7`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:23:06,736::task::595::TaskManager.Task::(_updateState) Task=`f99a8c06-f1e9-4bd5-88fa-647872bd96f7`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:23:06,736::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:23:06,736::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:23:06,736::task::990::TaskManager.Task::(_decref) Task=`f99a8c06-f1e9-4bd5-88fa-647872bd96f7`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:23:06,741::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3c71a9dd] >Thread-338::DEBUG::2014-05-13 11:23:06,741::task::595::TaskManager.Task::(_updateState) Task=`15bad7f0-28c7-4cb0-8a21-d46ca8990734`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:23:06,741::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:23:06,742::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`512973de-e2ce-4467-aedd-ae5bd3e8b7de`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:23:06,742::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:23:06,742::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:23:06,742::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`512973de-e2ce-4467-aedd-ae5bd3e8b7de`::Granted request >Thread-338::DEBUG::2014-05-13 11:23:06,743::task::827::TaskManager.Task::(resourceAcquired) Task=`15bad7f0-28c7-4cb0-8a21-d46ca8990734`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:23:06,743::task::990::TaskManager.Task::(_decref) Task=`15bad7f0-28c7-4cb0-8a21-d46ca8990734`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:23:06,744::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:23:06,744::task::1185::TaskManager.Task::(prepare) Task=`15bad7f0-28c7-4cb0-8a21-d46ca8990734`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:23:06,744::task::595::TaskManager.Task::(_updateState) Task=`15bad7f0-28c7-4cb0-8a21-d46ca8990734`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:23:06,744::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:23:06,744::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:23:06,745::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:23:06,745::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:23:06,745::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:23:06,745::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:23:06,745::task::990::TaskManager.Task::(_decref) Task=`15bad7f0-28c7-4cb0-8a21-d46ca8990734`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:23:07,175::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:23:07,260::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036829 s, 27.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:23:08,512::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:23:09,273::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:23:09,351::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0351752 s, 29.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:23:10,247::task::595::TaskManager.Task::(_updateState) Task=`153488dd-3a11-4a09-bc1b-5ac607f31f4f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:23:10,247::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:23:10,248::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000665588', 'lastCheck': '5.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:23:10,248::task::1185::TaskManager.Task::(prepare) Task=`153488dd-3a11-4a09-bc1b-5ac607f31f4f`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000665588', 'lastCheck': '5.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:23:10,248::task::595::TaskManager.Task::(_updateState) Task=`153488dd-3a11-4a09-bc1b-5ac607f31f4f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:23:10,248::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:23:10,248::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:23:10,249::task::990::TaskManager.Task::(_decref) Task=`153488dd-3a11-4a09-bc1b-5ac607f31f4f`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:23:11,364::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:23:11,531::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:23:11,738::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.330545 s, 3.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:23:13,751::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:23:13,841::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0326941 s, 31.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:23:14,552::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:23:15,269::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:23:15,278::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000648954 s, 6.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:23:15,854::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:23:15,934::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373751 s, 27.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:23:16,763::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:23:16,764::task::595::TaskManager.Task::(_updateState) Task=`4cbb3269-8a18-4232-a857-f4c106050b46`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:23:16,764::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:23:16,764::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:23:16,764::task::1185::TaskManager.Task::(prepare) Task=`4cbb3269-8a18-4232-a857-f4c106050b46`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:23:16,765::task::595::TaskManager.Task::(_updateState) Task=`4cbb3269-8a18-4232-a857-f4c106050b46`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:23:16,765::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:23:16,765::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:23:16,765::task::990::TaskManager.Task::(_decref) Task=`4cbb3269-8a18-4232-a857-f4c106050b46`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:23:16,771::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:23:16,771::task::595::TaskManager.Task::(_updateState) Task=`9420e6b1-08f6-4be6-ae01-ee6c0dc33473`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:23:16,771::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:23:16,772::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d2633d77-8686-4cfb-b182-295e8e96872e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:23:16,772::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:23:16,772::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:23:16,772::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d2633d77-8686-4cfb-b182-295e8e96872e`::Granted request >Thread-338::DEBUG::2014-05-13 11:23:16,772::task::827::TaskManager.Task::(resourceAcquired) Task=`9420e6b1-08f6-4be6-ae01-ee6c0dc33473`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:23:16,773::task::990::TaskManager.Task::(_decref) Task=`9420e6b1-08f6-4be6-ae01-ee6c0dc33473`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:23:16,773::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:23:16,773::task::1185::TaskManager.Task::(prepare) Task=`9420e6b1-08f6-4be6-ae01-ee6c0dc33473`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:23:16,773::task::595::TaskManager.Task::(_updateState) Task=`9420e6b1-08f6-4be6-ae01-ee6c0dc33473`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:23:16,774::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:23:16,774::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:23:16,774::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:23:16,774::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:23:16,774::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:23:16,774::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:23:16,775::task::990::TaskManager.Task::(_decref) Task=`9420e6b1-08f6-4be6-ae01-ee6c0dc33473`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:23:17,572::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:23:17,947::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:23:18,022::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0334517 s, 30.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:23:20,035::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:23:20,109::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319782 s, 32.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:23:20,592::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:23:22,122::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:23:22,202::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374484 s, 27.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:23:23,612::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:23:24,215::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:23:24,291::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0329587 s, 31.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:23:25,285::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:23:25,294::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000588133 s, 7.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:23:25,396::task::595::TaskManager.Task::(_updateState) Task=`c1e9a7f1-d8d4-4b9d-aee9-d12da5b637b0`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:23:25,396::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:23:25,396::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000588133', 'lastCheck': '0.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:23:25,397::task::1185::TaskManager.Task::(prepare) Task=`c1e9a7f1-d8d4-4b9d-aee9-d12da5b637b0`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000588133', 'lastCheck': '0.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:23:25,397::task::595::TaskManager.Task::(_updateState) Task=`c1e9a7f1-d8d4-4b9d-aee9-d12da5b637b0`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:23:25,397::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:23:25,397::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:23:25,397::task::990::TaskManager.Task::(_decref) Task=`c1e9a7f1-d8d4-4b9d-aee9-d12da5b637b0`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:23:26,304::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:23:26,394::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0464974 s, 22.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:23:26,630::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:23:26,795::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:23:26,795::task::595::TaskManager.Task::(_updateState) Task=`dcb29911-d8b9-4e4a-bfb8-5fee540c29ec`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:23:26,796::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:23:26,796::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:23:26,796::task::1185::TaskManager.Task::(prepare) Task=`dcb29911-d8b9-4e4a-bfb8-5fee540c29ec`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:23:26,796::task::595::TaskManager.Task::(_updateState) Task=`dcb29911-d8b9-4e4a-bfb8-5fee540c29ec`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:23:26,797::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:23:26,797::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:23:26,797::task::990::TaskManager.Task::(_decref) Task=`dcb29911-d8b9-4e4a-bfb8-5fee540c29ec`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:23:26,801::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:23:26,802::task::595::TaskManager.Task::(_updateState) Task=`74dc6688-3e18-409f-9324-1fc1d2763c05`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:23:26,802::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:23:26,802::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fb0fb8f4-0baf-4d9b-8308-cf116bfa3d63`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:23:26,802::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:23:26,803::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:23:26,803::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fb0fb8f4-0baf-4d9b-8308-cf116bfa3d63`::Granted request >Thread-338::DEBUG::2014-05-13 11:23:26,803::task::827::TaskManager.Task::(resourceAcquired) Task=`74dc6688-3e18-409f-9324-1fc1d2763c05`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:23:26,803::task::990::TaskManager.Task::(_decref) Task=`74dc6688-3e18-409f-9324-1fc1d2763c05`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:23:26,804::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:23:26,804::task::1185::TaskManager.Task::(prepare) Task=`74dc6688-3e18-409f-9324-1fc1d2763c05`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:23:26,804::task::595::TaskManager.Task::(_updateState) Task=`74dc6688-3e18-409f-9324-1fc1d2763c05`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:23:26,805::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:23:26,805::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:23:26,805::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:23:26,805::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:23:26,805::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:23:26,806::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:23:26,806::task::990::TaskManager.Task::(_decref) Task=`74dc6688-3e18-409f-9324-1fc1d2763c05`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:23:28,407::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:23:28,490::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0351241 s, 29.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:23:29,649::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:23:30,502::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:23:30,577::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309538 s, 33.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:23:32,590::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:23:32,666::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031632 s, 32.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:23:32,667::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:23:34,682::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:23:34,760::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0349121 s, 29.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:23:35,300::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:23:35,309::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00058608 s, 7.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:23:35,685::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:23:36,774::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 11:23:36,824::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:23:36,826::task::595::TaskManager.Task::(_updateState) Task=`a7ae11a9-e1fe-4989-9648-4b860e25744c`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:23:36,827::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:23:36,829::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:23:36,830::task::1185::TaskManager.Task::(prepare) Task=`a7ae11a9-e1fe-4989-9648-4b860e25744c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:23:36,831::task::595::TaskManager.Task::(_updateState) Task=`a7ae11a9-e1fe-4989-9648-4b860e25744c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:23:36,831::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:23:36,832::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:23:36,833::task::990::TaskManager.Task::(_decref) Task=`a7ae11a9-e1fe-4989-9648-4b860e25744c`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:23:36,841::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:23:36,843::task::595::TaskManager.Task::(_updateState) Task=`3a917a61-a3ce-472f-a7cd-3effe24443fa`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:23:36,844::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:23:36,846::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9176c351-3772-4ebe-bdf4-0735763ad605`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:23:36,847::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:23:36,848::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:23:36,849::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9176c351-3772-4ebe-bdf4-0735763ad605`::Granted request >Thread-338::DEBUG::2014-05-13 11:23:36,851::task::827::TaskManager.Task::(resourceAcquired) Task=`3a917a61-a3ce-472f-a7cd-3effe24443fa`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:23:36,852::task::990::TaskManager.Task::(_decref) Task=`3a917a61-a3ce-472f-a7cd-3effe24443fa`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:23:36,856::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:23:36,857::task::1185::TaskManager.Task::(prepare) Task=`3a917a61-a3ce-472f-a7cd-3effe24443fa`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:23:36,857::task::595::TaskManager.Task::(_updateState) Task=`3a917a61-a3ce-472f-a7cd-3effe24443fa`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:23:36,858::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:23:36,859::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:23:36,860::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:23:36,862::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:23:36,862::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:23:36,863::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:23:36,865::task::990::TaskManager.Task::(_decref) Task=`3a917a61-a3ce-472f-a7cd-3effe24443fa`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:23:36,892::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374711 s, 27.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:23:38,705::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:23:38,905::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:23:38,990::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0430761 s, 23.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:23:40,530::task::595::TaskManager.Task::(_updateState) Task=`acc16389-ce81-4b57-8539-0842edb3fecc`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:23:40,530::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:23:40,531::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00058608', 'lastCheck': '5.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:23:40,531::task::1185::TaskManager.Task::(prepare) Task=`acc16389-ce81-4b57-8539-0842edb3fecc`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00058608', 'lastCheck': '5.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:23:40,531::task::595::TaskManager.Task::(_updateState) Task=`acc16389-ce81-4b57-8539-0842edb3fecc`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:23:40,531::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:23:40,531::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:23:40,531::task::990::TaskManager.Task::(_decref) Task=`acc16389-ce81-4b57-8539-0842edb3fecc`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:23:41,003::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:23:41,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303656 s, 33.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:23:41,742::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:23:43,090::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:23:43,169::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358572 s, 28.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:23:44,760::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:23:45,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:23:45,256::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303238 s, 33.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:23:45,317::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:23:45,325::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000591269 s, 6.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:23:46,888::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4a63f6c8] >Thread-13::DEBUG::2014-05-13 11:23:46,888::task::595::TaskManager.Task::(_updateState) Task=`99e58343-5da5-44f8-bbb9-72ed1b7a148d`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:23:46,889::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:23:46,889::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:23:46,889::task::1185::TaskManager.Task::(prepare) Task=`99e58343-5da5-44f8-bbb9-72ed1b7a148d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:23:46,889::task::595::TaskManager.Task::(_updateState) Task=`99e58343-5da5-44f8-bbb9-72ed1b7a148d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:23:46,889::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:23:46,889::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:23:46,890::task::990::TaskManager.Task::(_decref) Task=`99e58343-5da5-44f8-bbb9-72ed1b7a148d`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:23:46,895::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4a63f6c8] >Thread-338::DEBUG::2014-05-13 11:23:46,895::task::595::TaskManager.Task::(_updateState) Task=`2a293b79-bf40-4a25-a77d-304dc2258b86`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:23:46,895::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:23:46,896::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b202d6e1-fe70-4930-bd01-103ec3d5dfef`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:23:46,896::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:23:46,896::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:23:46,896::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b202d6e1-fe70-4930-bd01-103ec3d5dfef`::Granted request >Thread-338::DEBUG::2014-05-13 11:23:46,897::task::827::TaskManager.Task::(resourceAcquired) Task=`2a293b79-bf40-4a25-a77d-304dc2258b86`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:23:46,898::task::990::TaskManager.Task::(_decref) Task=`2a293b79-bf40-4a25-a77d-304dc2258b86`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:23:46,898::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:23:46,898::task::1185::TaskManager.Task::(prepare) Task=`2a293b79-bf40-4a25-a77d-304dc2258b86`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:23:46,899::task::595::TaskManager.Task::(_updateState) Task=`2a293b79-bf40-4a25-a77d-304dc2258b86`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:23:46,899::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:23:46,899::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:23:46,899::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:23:46,899::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:23:46,899::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:23:46,900::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:23:46,900::task::990::TaskManager.Task::(_decref) Task=`2a293b79-bf40-4a25-a77d-304dc2258b86`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:23:47,270::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:23:47,345::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316091 s, 32.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:23:47,779::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:23:49,358::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:23:49,437::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358674 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:23:50,798::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:23:51,450::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:23:51,823::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.331068 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:23:53,816::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:23:53,836::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:23:53,930::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323367 s, 31.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:23:55,332::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:23:55,341::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000650886 s, 6.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:23:55,661::task::595::TaskManager.Task::(_updateState) Task=`c244b020-3674-44cd-8367-c6a4149dc6ee`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:23:55,661::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:23:55,661::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000650886', 'lastCheck': '0.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:23:55,661::task::1185::TaskManager.Task::(prepare) Task=`c244b020-3674-44cd-8367-c6a4149dc6ee`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000650886', 'lastCheck': '0.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:23:55,661::task::595::TaskManager.Task::(_updateState) Task=`c244b020-3674-44cd-8367-c6a4149dc6ee`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:23:55,662::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:23:55,662::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:23:55,662::task::990::TaskManager.Task::(_decref) Task=`c244b020-3674-44cd-8367-c6a4149dc6ee`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:23:55,943::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:23:56,021::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352597 s, 29.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:23:56,835::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:23:56,921::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:23:56,921::task::595::TaskManager.Task::(_updateState) Task=`caa9ec18-5ab3-4b69-b0d1-3fba14f35b68`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:23:56,921::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:23:56,922::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:23:56,922::task::1185::TaskManager.Task::(prepare) Task=`caa9ec18-5ab3-4b69-b0d1-3fba14f35b68`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:23:56,922::task::595::TaskManager.Task::(_updateState) Task=`caa9ec18-5ab3-4b69-b0d1-3fba14f35b68`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:23:56,922::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:23:56,922::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:23:56,923::task::990::TaskManager.Task::(_decref) Task=`caa9ec18-5ab3-4b69-b0d1-3fba14f35b68`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:23:56,929::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:23:56,930::task::595::TaskManager.Task::(_updateState) Task=`12f6d327-accb-4db8-b138-aad04dfee152`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:23:56,930::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:23:56,930::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ca594229-a068-4e1b-a3c4-08b8cba3c636`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:23:56,930::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:23:56,931::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:23:56,931::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ca594229-a068-4e1b-a3c4-08b8cba3c636`::Granted request >Thread-338::DEBUG::2014-05-13 11:23:56,931::task::827::TaskManager.Task::(resourceAcquired) Task=`12f6d327-accb-4db8-b138-aad04dfee152`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:23:56,931::task::990::TaskManager.Task::(_decref) Task=`12f6d327-accb-4db8-b138-aad04dfee152`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:23:56,932::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:23:56,932::task::1185::TaskManager.Task::(prepare) Task=`12f6d327-accb-4db8-b138-aad04dfee152`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:23:56,932::task::595::TaskManager.Task::(_updateState) Task=`12f6d327-accb-4db8-b138-aad04dfee152`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:23:56,933::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:23:56,933::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:23:56,933::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:23:56,933::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:23:56,933::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:23:56,934::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:23:56,934::task::990::TaskManager.Task::(_decref) Task=`12f6d327-accb-4db8-b138-aad04dfee152`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:23:58,034::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:23:58,112::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0350158 s, 29.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:23:59,853::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:24:00,125::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:00,199::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303941 s, 33.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:24:02,212::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:02,285::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0301466 s, 34.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:24:02,874::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:24:04,297::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:04,375::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0347723 s, 29.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:24:05,348::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:24:05,356::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00065564 s, 6.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:24:05,892::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:24:06,388::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:06,480::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0476082 s, 21.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:24:06,955::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6623fb10] >Thread-13::DEBUG::2014-05-13 11:24:06,955::task::595::TaskManager.Task::(_updateState) Task=`90ac2e90-e9cb-4e2d-a887-0787791f94a0`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:24:06,956::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:24:06,956::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:24:06,956::task::1185::TaskManager.Task::(prepare) Task=`90ac2e90-e9cb-4e2d-a887-0787791f94a0`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:24:06,956::task::595::TaskManager.Task::(_updateState) Task=`90ac2e90-e9cb-4e2d-a887-0787791f94a0`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:24:06,956::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:24:06,957::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:24:06,957::task::990::TaskManager.Task::(_decref) Task=`90ac2e90-e9cb-4e2d-a887-0787791f94a0`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:24:06,961::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6623fb10] >Thread-338::DEBUG::2014-05-13 11:24:06,962::task::595::TaskManager.Task::(_updateState) Task=`6fb170e7-bab8-47da-999e-13c3c69f113a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:24:06,962::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:24:06,962::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`cf953474-da1f-45f7-be61-9ca156d3a630`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:24:06,963::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:24:06,963::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:24:06,963::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`cf953474-da1f-45f7-be61-9ca156d3a630`::Granted request >Thread-338::DEBUG::2014-05-13 11:24:06,963::task::827::TaskManager.Task::(resourceAcquired) Task=`6fb170e7-bab8-47da-999e-13c3c69f113a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:24:06,964::task::990::TaskManager.Task::(_decref) Task=`6fb170e7-bab8-47da-999e-13c3c69f113a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:24:06,964::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:24:06,964::task::1185::TaskManager.Task::(prepare) Task=`6fb170e7-bab8-47da-999e-13c3c69f113a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:24:06,965::task::595::TaskManager.Task::(_updateState) Task=`6fb170e7-bab8-47da-999e-13c3c69f113a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:24:06,965::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:24:06,965::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:24:06,965::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:24:06,965::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:24:06,966::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:24:06,966::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:24:06,966::task::990::TaskManager.Task::(_decref) Task=`6fb170e7-bab8-47da-999e-13c3c69f113a`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:24:08,492::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:08,566::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0299295 s, 34.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:24:08,911::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:24:10,579::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:10,657::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0351162 s, 29.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:24:10,792::task::595::TaskManager.Task::(_updateState) Task=`6663accb-bc53-4122-870c-579beb81ef37`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:24:10,792::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:24:10,793::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00065564', 'lastCheck': '5.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:24:10,793::task::1185::TaskManager.Task::(prepare) Task=`6663accb-bc53-4122-870c-579beb81ef37`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00065564', 'lastCheck': '5.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:24:10,793::task::595::TaskManager.Task::(_updateState) Task=`6663accb-bc53-4122-870c-579beb81ef37`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:24:10,793::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:24:10,793::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:24:10,793::task::990::TaskManager.Task::(_decref) Task=`6663accb-bc53-4122-870c-579beb81ef37`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:24:11,932::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:24:12,670::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:12,749::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362276 s, 28.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:24:14,762::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:14,839::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358172 s, 28.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:24:14,950::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:24:15,363::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:24:15,371::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00055118 s, 7.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:24:16,852::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:16,929::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.034147 s, 30.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:24:16,983::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5affc0d0] >Thread-13::DEBUG::2014-05-13 11:24:16,983::task::595::TaskManager.Task::(_updateState) Task=`4625cc3a-6a3f-4a4e-97fc-441be700c9aa`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:24:16,984::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:24:16,984::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:24:16,984::task::1185::TaskManager.Task::(prepare) Task=`4625cc3a-6a3f-4a4e-97fc-441be700c9aa`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:24:16,984::task::595::TaskManager.Task::(_updateState) Task=`4625cc3a-6a3f-4a4e-97fc-441be700c9aa`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:24:16,985::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:24:16,985::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:24:16,985::task::990::TaskManager.Task::(_decref) Task=`4625cc3a-6a3f-4a4e-97fc-441be700c9aa`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:24:16,989::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5affc0d0] >Thread-338::DEBUG::2014-05-13 11:24:16,989::task::595::TaskManager.Task::(_updateState) Task=`6701f3c7-2c0f-4e84-9537-087e165292fb`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:24:16,990::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:24:16,990::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e84496a1-4a20-4d2a-9e93-a6d7aa18738a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:24:16,990::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:24:16,991::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:24:16,991::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e84496a1-4a20-4d2a-9e93-a6d7aa18738a`::Granted request >Thread-338::DEBUG::2014-05-13 11:24:16,991::task::827::TaskManager.Task::(resourceAcquired) Task=`6701f3c7-2c0f-4e84-9537-087e165292fb`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:24:16,991::task::990::TaskManager.Task::(_decref) Task=`6701f3c7-2c0f-4e84-9537-087e165292fb`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:24:16,992::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:24:16,992::task::1185::TaskManager.Task::(prepare) Task=`6701f3c7-2c0f-4e84-9537-087e165292fb`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:24:16,992::task::595::TaskManager.Task::(_updateState) Task=`6701f3c7-2c0f-4e84-9537-087e165292fb`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:24:16,993::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:24:16,993::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:24:16,993::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:24:16,993::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:24:16,993::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:24:16,994::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:24:16,994::task::990::TaskManager.Task::(_decref) Task=`6701f3c7-2c0f-4e84-9537-087e165292fb`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:24:17,970::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:24:18,942::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:19,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0481479 s, 21.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:24:20,987::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:24:21,045::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:21,122::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0330294 s, 31.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:24:23,135::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:23,210::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0324847 s, 31.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:24:24,008::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:24:25,223::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:25,302::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0368409 s, 27.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:24:25,379::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:24:25,387::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000651006 s, 6.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:24:25,934::task::595::TaskManager.Task::(_updateState) Task=`45771dd3-2694-450c-aa8d-3cea1c04f079`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:24:25,934::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:24:25,934::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000651006', 'lastCheck': '0.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:24:25,934::task::1185::TaskManager.Task::(prepare) Task=`45771dd3-2694-450c-aa8d-3cea1c04f079`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000651006', 'lastCheck': '0.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:24:25,934::task::595::TaskManager.Task::(_updateState) Task=`45771dd3-2694-450c-aa8d-3cea1c04f079`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:24:25,934::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:24:25,935::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:24:25,935::task::990::TaskManager.Task::(_decref) Task=`45771dd3-2694-450c-aa8d-3cea1c04f079`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 11:24:27,012::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [45b9a91f] >Thread-13::DEBUG::2014-05-13 11:24:27,013::task::595::TaskManager.Task::(_updateState) Task=`003a13a3-e8de-4862-b809-f5488220df2f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:24:27,013::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:24:27,013::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:24:27,013::task::1185::TaskManager.Task::(prepare) Task=`003a13a3-e8de-4862-b809-f5488220df2f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:24:27,014::task::595::TaskManager.Task::(_updateState) Task=`003a13a3-e8de-4862-b809-f5488220df2f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:24:27,014::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:24:27,014::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:24:27,014::task::990::TaskManager.Task::(_decref) Task=`003a13a3-e8de-4862-b809-f5488220df2f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:24:27,018::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [45b9a91f] >Thread-338::DEBUG::2014-05-13 11:24:27,019::task::595::TaskManager.Task::(_updateState) Task=`c81b4352-ef20-4a84-bec2-b576fb7d4448`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:24:27,019::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:24:27,019::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c100a497-dadf-42c5-bc18-fd9a12b0b0e0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:24:27,020::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:24:27,020::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:24:27,020::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c100a497-dadf-42c5-bc18-fd9a12b0b0e0`::Granted request >Thread-338::DEBUG::2014-05-13 11:24:27,020::task::827::TaskManager.Task::(resourceAcquired) Task=`c81b4352-ef20-4a84-bec2-b576fb7d4448`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:24:27,021::task::990::TaskManager.Task::(_decref) Task=`c81b4352-ef20-4a84-bec2-b576fb7d4448`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:24:27,021::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:24:27,022::task::1185::TaskManager.Task::(prepare) Task=`c81b4352-ef20-4a84-bec2-b576fb7d4448`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:24:27,022::task::595::TaskManager.Task::(_updateState) Task=`c81b4352-ef20-4a84-bec2-b576fb7d4448`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:24:27,022::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:24:27,022::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:24:27,022::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:24:27,023::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:24:27,023::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:24:27,023::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:24:27,023::task::990::TaskManager.Task::(_decref) Task=`c81b4352-ef20-4a84-bec2-b576fb7d4448`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:24:27,034::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:24:27,315::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:27,396::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0377299 s, 27.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:24:29,410::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:29,483::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0300263 s, 34.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:24:30,053::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:24:31,496::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:31,575::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358355 s, 28.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:24:33,073::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:24:33,588::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:33,948::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.316552 s, 3.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:24:35,394::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:24:35,403::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000649878 s, 6.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:24:35,961::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:36,058::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0430405 s, 23.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:24:36,092::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:24:37,043::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:24:37,043::task::595::TaskManager.Task::(_updateState) Task=`e8328152-e5b2-4e83-9786-eec6e01c5f3d`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:24:37,043::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:24:37,044::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:24:37,044::task::1185::TaskManager.Task::(prepare) Task=`e8328152-e5b2-4e83-9786-eec6e01c5f3d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:24:37,044::task::595::TaskManager.Task::(_updateState) Task=`e8328152-e5b2-4e83-9786-eec6e01c5f3d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:24:37,044::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:24:37,045::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:24:37,045::task::990::TaskManager.Task::(_decref) Task=`e8328152-e5b2-4e83-9786-eec6e01c5f3d`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:24:37,052::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:24:37,052::task::595::TaskManager.Task::(_updateState) Task=`87ccfd1e-9f77-4b1e-95fd-a56289b33b73`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:24:37,052::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:24:37,053::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8cea76a9-8084-430a-815c-6bd21c94ec68`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:24:37,053::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:24:37,053::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:24:37,053::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8cea76a9-8084-430a-815c-6bd21c94ec68`::Granted request >Thread-338::DEBUG::2014-05-13 11:24:37,054::task::827::TaskManager.Task::(resourceAcquired) Task=`87ccfd1e-9f77-4b1e-95fd-a56289b33b73`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:24:37,054::task::990::TaskManager.Task::(_decref) Task=`87ccfd1e-9f77-4b1e-95fd-a56289b33b73`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:24:37,055::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:24:37,055::task::1185::TaskManager.Task::(prepare) Task=`87ccfd1e-9f77-4b1e-95fd-a56289b33b73`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:24:37,055::task::595::TaskManager.Task::(_updateState) Task=`87ccfd1e-9f77-4b1e-95fd-a56289b33b73`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:24:37,055::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:24:37,055::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:24:37,056::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:24:37,056::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:24:37,056::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:24:37,056::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:24:37,056::task::990::TaskManager.Task::(_decref) Task=`87ccfd1e-9f77-4b1e-95fd-a56289b33b73`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:24:38,071::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:38,146::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318548 s, 32.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:24:39,112::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:24:40,159::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:40,239::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036817 s, 27.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:24:41,067::task::595::TaskManager.Task::(_updateState) Task=`604a6b6d-1c5a-4f77-a230-0c2fe487ecda`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:24:41,067::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:24:41,067::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000649878', 'lastCheck': '5.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:24:41,068::task::1185::TaskManager.Task::(prepare) Task=`604a6b6d-1c5a-4f77-a230-0c2fe487ecda`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000649878', 'lastCheck': '5.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:24:41,068::task::595::TaskManager.Task::(_updateState) Task=`604a6b6d-1c5a-4f77-a230-0c2fe487ecda`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:24:41,068::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:24:41,068::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:24:41,068::task::990::TaskManager.Task::(_decref) Task=`604a6b6d-1c5a-4f77-a230-0c2fe487ecda`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:24:42,132::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:24:42,252::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:42,331::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355932 s, 28.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:24:44,345::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:44,419::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305025 s, 33.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:24:45,152::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:24:45,410::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:24:45,438::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000565207 s, 7.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:24:46,432::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:46,847::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.372967 s, 2.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:24:47,075::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:24:47,076::task::595::TaskManager.Task::(_updateState) Task=`3c91bdf6-e600-41bd-8299-beed5f4a148b`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:24:47,076::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:24:47,076::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:24:47,076::task::1185::TaskManager.Task::(prepare) Task=`3c91bdf6-e600-41bd-8299-beed5f4a148b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:24:47,076::task::595::TaskManager.Task::(_updateState) Task=`3c91bdf6-e600-41bd-8299-beed5f4a148b`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:24:47,077::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:24:47,077::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:24:47,077::task::990::TaskManager.Task::(_decref) Task=`3c91bdf6-e600-41bd-8299-beed5f4a148b`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:24:47,082::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:24:47,082::task::595::TaskManager.Task::(_updateState) Task=`31f37c5a-c4f9-4978-b821-8bf24c2a4d6d`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:24:47,082::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:24:47,083::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`00a66e1b-f95e-4da3-a071-e857f43a2d59`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:24:47,083::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:24:47,083::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:24:47,083::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`00a66e1b-f95e-4da3-a071-e857f43a2d59`::Granted request >Thread-338::DEBUG::2014-05-13 11:24:47,083::task::827::TaskManager.Task::(resourceAcquired) Task=`31f37c5a-c4f9-4978-b821-8bf24c2a4d6d`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:24:47,084::task::990::TaskManager.Task::(_decref) Task=`31f37c5a-c4f9-4978-b821-8bf24c2a4d6d`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:24:47,084::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:24:47,084::task::1185::TaskManager.Task::(prepare) Task=`31f37c5a-c4f9-4978-b821-8bf24c2a4d6d`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:24:47,085::task::595::TaskManager.Task::(_updateState) Task=`31f37c5a-c4f9-4978-b821-8bf24c2a4d6d`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:24:47,085::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:24:47,085::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:24:47,085::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:24:47,085::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:24:47,085::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:24:47,085::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:24:47,086::task::990::TaskManager.Task::(_decref) Task=`31f37c5a-c4f9-4978-b821-8bf24c2a4d6d`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:24:48,173::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:24:48,860::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:48,933::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0302508 s, 33.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:24:50,946::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:51,022::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0335566 s, 30.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:24:51,192::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:24:53,035::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:53,109::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0302625 s, 33.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:24:54,213::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:24:55,122::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:55,200::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0346591 s, 29.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:24:55,443::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 41830dad-f012-4b4d-b636-863f66d5ca4b >Thread-322::DEBUG::2014-05-13 11:24:55,443::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-322::DEBUG::2014-05-13 11:24:55,446::blockSD::336::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:24:55,461::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.00792989 s, 258 kB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:24:55,462::misc::287::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.00792989 s, 258 kB/s'], size: 2048 >Thread-322::DEBUG::2014-05-13 11:24:55,462::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-322::WARNING::2014-05-13 11:24:55,462::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-322::DEBUG::2014-05-13 11:24:55,462::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-322::DEBUG::2014-05-13 11:24:55,462::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:24:55,463::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:24:55,463::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:24:55,463::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:24:55,463::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:24:55,464::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 11:24:55,723::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:24:55,725::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:24:55,725::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'LOCKPOLICY=', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', 'DESCRIPTION=ISCSI', 'LOGBLKSIZE=512', 'TYPE=ISCSI', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'PHYBLKSIZE=512', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', 'CLASS=Data', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'POOL_DESCRIPTION=DC_2', 'POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'MASTER_VERSION=1', 'ROLE=Master', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', '_SHA_CKSUM=28bb2d7cc60e0d779eb69fe93d050c16c769f069', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0'] >Thread-322::INFO::2014-05-13 11:24:55,726::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_imageNS already registered >Thread-322::INFO::2014-05-13 11:24:55,726::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_volumeNS already registered >Thread-322::INFO::2014-05-13 11:24:55,726::blockSD::452::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_lvmActivationNS already registered >Thread-322::DEBUG::2014-05-13 11:24:55,727::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 11:24:55,895::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:24:55,899::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:24:55,908::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000802423 s, 5.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:24:56,204::task::595::TaskManager.Task::(_updateState) Task=`52519079-0258-47c4-a612-608bc6432daa`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:24:56,223::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:24:56,223::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000802423', 'lastCheck': '0.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:24:56,223::task::1185::TaskManager.Task::(prepare) Task=`52519079-0258-47c4-a612-608bc6432daa`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000802423', 'lastCheck': '0.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:24:56,223::task::595::TaskManager.Task::(_updateState) Task=`52519079-0258-47c4-a612-608bc6432daa`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:24:56,223::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:24:56,224::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:24:56,224::task::990::TaskManager.Task::(_decref) Task=`52519079-0258-47c4-a612-608bc6432daa`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 11:24:57,103::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:24:57,104::task::595::TaskManager.Task::(_updateState) Task=`837f8078-199e-49af-a889-2af019291cae`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:24:57,104::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:24:57,104::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:24:57,105::task::1185::TaskManager.Task::(prepare) Task=`837f8078-199e-49af-a889-2af019291cae`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:24:57,105::task::595::TaskManager.Task::(_updateState) Task=`837f8078-199e-49af-a889-2af019291cae`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:24:57,105::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:24:57,105::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:24:57,105::task::990::TaskManager.Task::(_decref) Task=`837f8078-199e-49af-a889-2af019291cae`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:24:57,110::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:24:57,110::task::595::TaskManager.Task::(_updateState) Task=`af3265c1-d650-4b71-848e-23954be8a192`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:24:57,110::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:24:57,112::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3db6ccf9-f077-430e-a02f-e518c554fa6d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:24:57,112::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:24:57,112::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:24:57,112::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3db6ccf9-f077-430e-a02f-e518c554fa6d`::Granted request >Thread-338::DEBUG::2014-05-13 11:24:57,113::task::827::TaskManager.Task::(resourceAcquired) Task=`af3265c1-d650-4b71-848e-23954be8a192`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:24:57,113::task::990::TaskManager.Task::(_decref) Task=`af3265c1-d650-4b71-848e-23954be8a192`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:24:57,114::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:24:57,114::task::1185::TaskManager.Task::(prepare) Task=`af3265c1-d650-4b71-848e-23954be8a192`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:24:57,114::task::595::TaskManager.Task::(_updateState) Task=`af3265c1-d650-4b71-848e-23954be8a192`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:24:57,114::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:24:57,114::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:24:57,115::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:24:57,115::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:24:57,115::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:24:57,115::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:24:57,115::task::990::TaskManager.Task::(_decref) Task=`af3265c1-d650-4b71-848e-23954be8a192`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:24:57,213::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:24:57,234::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:24:57,573::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.316242 s, 3.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:24:59,585::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:24:59,660::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310051 s, 33.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:25:00,253::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:25:01,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:01,751::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358722 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:25:03,274::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:25:03,764::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:03,838::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0298529 s, 34.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:25:05,850::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 11:25:05,935::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:25:05,954::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000653911 s, 6.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:25:05,980::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303601 s, 33.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:25:06,296::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:25:07,134::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5eff0b16] >Thread-13::DEBUG::2014-05-13 11:25:07,171::task::595::TaskManager.Task::(_updateState) Task=`932ee4b0-53e5-457d-bc3f-bd2e598cd54f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:25:07,171::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:25:07,171::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:25:07,171::task::1185::TaskManager.Task::(prepare) Task=`932ee4b0-53e5-457d-bc3f-bd2e598cd54f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:25:07,172::task::595::TaskManager.Task::(_updateState) Task=`932ee4b0-53e5-457d-bc3f-bd2e598cd54f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:25:07,172::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:25:07,172::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:25:07,172::task::990::TaskManager.Task::(_decref) Task=`932ee4b0-53e5-457d-bc3f-bd2e598cd54f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:25:07,177::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5eff0b16] >Thread-338::DEBUG::2014-05-13 11:25:07,177::task::595::TaskManager.Task::(_updateState) Task=`578d9ed2-5f56-4904-9f94-a25f25eed9ec`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:25:07,178::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:25:07,178::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3443d157-056e-487a-a2b6-e8d223089e83`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:25:07,178::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:25:07,179::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:25:07,179::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3443d157-056e-487a-a2b6-e8d223089e83`::Granted request >Thread-338::DEBUG::2014-05-13 11:25:07,179::task::827::TaskManager.Task::(resourceAcquired) Task=`578d9ed2-5f56-4904-9f94-a25f25eed9ec`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:25:07,179::task::990::TaskManager.Task::(_decref) Task=`578d9ed2-5f56-4904-9f94-a25f25eed9ec`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:25:07,180::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:25:07,180::task::1185::TaskManager.Task::(prepare) Task=`578d9ed2-5f56-4904-9f94-a25f25eed9ec`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:25:07,180::task::595::TaskManager.Task::(_updateState) Task=`578d9ed2-5f56-4904-9f94-a25f25eed9ec`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:25:07,181::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:25:07,181::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:25:07,181::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:25:07,181::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:25:07,181::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:25:07,182::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:25:07,182::task::990::TaskManager.Task::(_decref) Task=`578d9ed2-5f56-4904-9f94-a25f25eed9ec`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:25:07,993::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:08,066::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305075 s, 33.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:25:09,316::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:25:10,079::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:10,453::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.329235 s, 3.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:25:11,371::task::595::TaskManager.Task::(_updateState) Task=`8bdfe3af-b51b-4c33-9238-e98944cfc0da`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:25:11,371::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:25:11,371::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000653911', 'lastCheck': '5.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:25:11,371::task::1185::TaskManager.Task::(prepare) Task=`8bdfe3af-b51b-4c33-9238-e98944cfc0da`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000653911', 'lastCheck': '5.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:25:11,372::task::595::TaskManager.Task::(_updateState) Task=`8bdfe3af-b51b-4c33-9238-e98944cfc0da`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:25:11,372::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:25:11,372::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:25:11,372::task::990::TaskManager.Task::(_decref) Task=`8bdfe3af-b51b-4c33-9238-e98944cfc0da`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:25:12,335::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:25:12,466::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:12,546::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352939 s, 29.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:25:14,559::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:14,633::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0299467 s, 34.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:25:15,351::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:25:15,975::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:25:15,983::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00054844 s, 7.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:25:16,646::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:16,724::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0346656 s, 29.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:25:17,205::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:25:17,205::task::595::TaskManager.Task::(_updateState) Task=`a3601559-1d60-4df4-b09c-53e009e31d7d`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:25:17,205::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:25:17,206::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:25:17,206::task::1185::TaskManager.Task::(prepare) Task=`a3601559-1d60-4df4-b09c-53e009e31d7d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:25:17,206::task::595::TaskManager.Task::(_updateState) Task=`a3601559-1d60-4df4-b09c-53e009e31d7d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:25:17,206::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:25:17,206::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:25:17,206::task::990::TaskManager.Task::(_decref) Task=`a3601559-1d60-4df4-b09c-53e009e31d7d`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:25:17,211::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:25:17,211::task::595::TaskManager.Task::(_updateState) Task=`f6808648-6898-4401-b25e-f14a2faaf968`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:25:17,211::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:25:17,212::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6e051865-5f55-4e07-8042-35614326eaec`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:25:17,212::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:25:17,212::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:25:17,212::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6e051865-5f55-4e07-8042-35614326eaec`::Granted request >Thread-338::DEBUG::2014-05-13 11:25:17,212::task::827::TaskManager.Task::(resourceAcquired) Task=`f6808648-6898-4401-b25e-f14a2faaf968`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:25:17,213::task::990::TaskManager.Task::(_decref) Task=`f6808648-6898-4401-b25e-f14a2faaf968`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:25:17,213::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:25:17,213::task::1185::TaskManager.Task::(prepare) Task=`f6808648-6898-4401-b25e-f14a2faaf968`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:25:17,213::task::595::TaskManager.Task::(_updateState) Task=`f6808648-6898-4401-b25e-f14a2faaf968`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:25:17,214::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:25:17,214::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:25:17,214::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:25:17,214::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:25:17,214::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:25:17,214::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:25:17,215::task::990::TaskManager.Task::(_decref) Task=`f6808648-6898-4401-b25e-f14a2faaf968`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:25:18,371::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:25:18,737::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:18,811::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309236 s, 33.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:25:20,824::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:20,898::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0297588 s, 34.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:25:21,391::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:25:22,911::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:22,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0448309 s, 22.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:25:24,418::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:25:25,012::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:25,090::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0345479 s, 29.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:25:25,990::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:25:25,999::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000780837 s, 5.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:25:26,500::task::595::TaskManager.Task::(_updateState) Task=`a80623b8-4972-4851-ae98-c65427fb9568`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:25:26,500::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:25:26,500::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000780837', 'lastCheck': '0.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:25:26,500::task::1185::TaskManager.Task::(prepare) Task=`a80623b8-4972-4851-ae98-c65427fb9568`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000780837', 'lastCheck': '0.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:25:26,501::task::595::TaskManager.Task::(_updateState) Task=`a80623b8-4972-4851-ae98-c65427fb9568`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:25:26,501::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:25:26,501::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:25:26,501::task::990::TaskManager.Task::(_decref) Task=`a80623b8-4972-4851-ae98-c65427fb9568`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:25:27,103::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:27,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035804 s, 28.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:25:27,238::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [19876d15] >Thread-13::DEBUG::2014-05-13 11:25:27,239::task::595::TaskManager.Task::(_updateState) Task=`b7861ecd-aaf6-4bf4-8993-d7e54155fa31`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:25:27,239::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:25:27,239::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:25:27,240::task::1185::TaskManager.Task::(prepare) Task=`b7861ecd-aaf6-4bf4-8993-d7e54155fa31`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:25:27,240::task::595::TaskManager.Task::(_updateState) Task=`b7861ecd-aaf6-4bf4-8993-d7e54155fa31`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:25:27,240::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:25:27,240::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:25:27,240::task::990::TaskManager.Task::(_decref) Task=`b7861ecd-aaf6-4bf4-8993-d7e54155fa31`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:25:27,245::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [19876d15] >Thread-338::DEBUG::2014-05-13 11:25:27,245::task::595::TaskManager.Task::(_updateState) Task=`8fad4956-fd3b-4eed-be88-9620f19faac5`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:25:27,245::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:25:27,246::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5ca54c06-d1d2-4cf6-bc26-d27ba8ab062c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:25:27,246::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:25:27,246::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:25:27,246::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5ca54c06-d1d2-4cf6-bc26-d27ba8ab062c`::Granted request >Thread-338::DEBUG::2014-05-13 11:25:27,246::task::827::TaskManager.Task::(resourceAcquired) Task=`8fad4956-fd3b-4eed-be88-9620f19faac5`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:25:27,247::task::990::TaskManager.Task::(_decref) Task=`8fad4956-fd3b-4eed-be88-9620f19faac5`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:25:27,247::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:25:27,248::task::1185::TaskManager.Task::(prepare) Task=`8fad4956-fd3b-4eed-be88-9620f19faac5`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:25:27,248::task::595::TaskManager.Task::(_updateState) Task=`8fad4956-fd3b-4eed-be88-9620f19faac5`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:25:27,248::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:25:27,248::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:25:27,248::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:25:27,249::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:25:27,249::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:25:27,249::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:25:27,249::task::990::TaskManager.Task::(_decref) Task=`8fad4956-fd3b-4eed-be88-9620f19faac5`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:25:27,441::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:25:29,195::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:29,268::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0302675 s, 33.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:25:30,462::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:25:31,281::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:31,360::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0368133 s, 27.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:25:33,373::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:33,448::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310055 s, 33.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:25:33,486::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:25:35,461::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:35,548::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0431607 s, 23.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:25:36,007::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:25:36,015::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000683985 s, 6.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:25:36,505::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:25:37,269::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:25:37,270::task::595::TaskManager.Task::(_updateState) Task=`80a5f653-6abd-4531-af94-71b2ff850d6c`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:25:37,270::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:25:37,270::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:25:37,271::task::1185::TaskManager.Task::(prepare) Task=`80a5f653-6abd-4531-af94-71b2ff850d6c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:25:37,271::task::595::TaskManager.Task::(_updateState) Task=`80a5f653-6abd-4531-af94-71b2ff850d6c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:25:37,271::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:25:37,271::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:25:37,271::task::990::TaskManager.Task::(_decref) Task=`80a5f653-6abd-4531-af94-71b2ff850d6c`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:25:37,276::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:25:37,276::task::595::TaskManager.Task::(_updateState) Task=`b8d339d0-d23f-44b9-af3f-344c60fb8a6b`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:25:37,276::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:25:37,277::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c5e991e7-e962-405c-b449-3c66cc6cb880`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:25:37,277::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:25:37,277::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:25:37,277::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c5e991e7-e962-405c-b449-3c66cc6cb880`::Granted request >Thread-338::DEBUG::2014-05-13 11:25:37,278::task::827::TaskManager.Task::(resourceAcquired) Task=`b8d339d0-d23f-44b9-af3f-344c60fb8a6b`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:25:37,278::task::990::TaskManager.Task::(_decref) Task=`b8d339d0-d23f-44b9-af3f-344c60fb8a6b`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:25:37,279::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:25:37,279::task::1185::TaskManager.Task::(prepare) Task=`b8d339d0-d23f-44b9-af3f-344c60fb8a6b`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:25:37,279::task::595::TaskManager.Task::(_updateState) Task=`b8d339d0-d23f-44b9-af3f-344c60fb8a6b`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:25:37,279::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:25:37,279::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:25:37,280::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:25:37,280::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:25:37,280::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:25:37,280::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:25:37,280::task::990::TaskManager.Task::(_decref) Task=`b8d339d0-d23f-44b9-af3f-344c60fb8a6b`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:25:37,561::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:37,640::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356457 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:25:39,526::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:25:39,653::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:39,729::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312856 s, 32.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:25:41,630::task::595::TaskManager.Task::(_updateState) Task=`dcb7ed9d-e4b3-471f-87d2-0b9c341b9cd0`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:25:41,631::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:25:41,631::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000683985', 'lastCheck': '5.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:25:41,631::task::1185::TaskManager.Task::(prepare) Task=`dcb7ed9d-e4b3-471f-87d2-0b9c341b9cd0`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000683985', 'lastCheck': '5.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:25:41,631::task::595::TaskManager.Task::(_updateState) Task=`dcb7ed9d-e4b3-471f-87d2-0b9c341b9cd0`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:25:41,631::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:25:41,631::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:25:41,631::task::990::TaskManager.Task::(_decref) Task=`dcb7ed9d-e4b3-471f-87d2-0b9c341b9cd0`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:25:41,742::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:41,821::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0349955 s, 29.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:25:42,546::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:25:43,834::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:43,915::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360552 s, 28.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:25:45,564::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:25:45,924::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:46,000::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314755 s, 32.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:25:46,023::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:25:46,031::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000636313 s, 6.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:25:47,299::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [303c64a0] >Thread-13::DEBUG::2014-05-13 11:25:47,300::task::595::TaskManager.Task::(_updateState) Task=`68595dbb-0d32-4bcc-806f-78a027dcc3a2`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:25:47,300::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:25:47,300::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:25:47,300::task::1185::TaskManager.Task::(prepare) Task=`68595dbb-0d32-4bcc-806f-78a027dcc3a2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:25:47,301::task::595::TaskManager.Task::(_updateState) Task=`68595dbb-0d32-4bcc-806f-78a027dcc3a2`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:25:47,301::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:25:47,301::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:25:47,301::task::990::TaskManager.Task::(_decref) Task=`68595dbb-0d32-4bcc-806f-78a027dcc3a2`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:25:47,306::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [303c64a0] >Thread-338::DEBUG::2014-05-13 11:25:47,307::task::595::TaskManager.Task::(_updateState) Task=`0613e374-fe8a-48f8-9406-6111120f747b`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:25:47,307::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:25:47,307::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`eca8864c-5179-4c55-a16b-6f8eb2c5c10a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:25:47,307::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:25:47,308::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:25:47,308::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`eca8864c-5179-4c55-a16b-6f8eb2c5c10a`::Granted request >Thread-338::DEBUG::2014-05-13 11:25:47,308::task::827::TaskManager.Task::(resourceAcquired) Task=`0613e374-fe8a-48f8-9406-6111120f747b`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:25:47,308::task::990::TaskManager.Task::(_decref) Task=`0613e374-fe8a-48f8-9406-6111120f747b`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:25:47,309::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:25:47,309::task::1185::TaskManager.Task::(prepare) Task=`0613e374-fe8a-48f8-9406-6111120f747b`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:25:47,309::task::595::TaskManager.Task::(_updateState) Task=`0613e374-fe8a-48f8-9406-6111120f747b`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:25:47,309::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:25:47,310::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:25:47,310::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:25:47,310::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:25:47,310::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:25:47,310::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:25:47,310::task::990::TaskManager.Task::(_decref) Task=`0613e374-fe8a-48f8-9406-6111120f747b`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:25:48,013::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:48,087::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.029953 s, 34.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:25:48,584::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:25:50,100::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:50,186::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0441932 s, 23.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:25:51,606::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:25:52,199::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:52,290::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0476728 s, 21.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:25:54,303::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:54,377::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310663 s, 33.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:25:54,627::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:25:56,038::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:25:56,046::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000594839 s, 6.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:25:56,391::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:56,470::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352572 s, 29.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:25:56,763::task::595::TaskManager.Task::(_updateState) Task=`23176f64-4d6c-49b5-85b3-a2c0e7e296d3`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:25:56,763::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:25:56,763::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000594839', 'lastCheck': '0.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:25:56,763::task::1185::TaskManager.Task::(prepare) Task=`23176f64-4d6c-49b5-85b3-a2c0e7e296d3`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000594839', 'lastCheck': '0.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:25:56,764::task::595::TaskManager.Task::(_updateState) Task=`23176f64-4d6c-49b5-85b3-a2c0e7e296d3`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:25:56,764::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:25:56,765::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:25:56,769::task::990::TaskManager.Task::(_decref) Task=`23176f64-4d6c-49b5-85b3-a2c0e7e296d3`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 11:25:57,330::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:25:57,341::task::595::TaskManager.Task::(_updateState) Task=`3e43617a-c7f1-4e6b-9c20-123bf26b9846`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:25:57,341::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:25:57,341::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:25:57,341::task::1185::TaskManager.Task::(prepare) Task=`3e43617a-c7f1-4e6b-9c20-123bf26b9846`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:25:57,342::task::595::TaskManager.Task::(_updateState) Task=`3e43617a-c7f1-4e6b-9c20-123bf26b9846`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:25:57,342::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:25:57,342::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:25:57,342::task::990::TaskManager.Task::(_decref) Task=`3e43617a-c7f1-4e6b-9c20-123bf26b9846`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:25:57,346::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:25:57,347::task::595::TaskManager.Task::(_updateState) Task=`175a1f4c-d1c8-4c0c-a169-4ac89b3047ea`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:25:57,347::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:25:57,347::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`98a791a2-db66-4b60-b2ff-a494f8f76cf3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:25:57,348::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:25:57,348::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:25:57,348::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`98a791a2-db66-4b60-b2ff-a494f8f76cf3`::Granted request >Thread-338::DEBUG::2014-05-13 11:25:57,348::task::827::TaskManager.Task::(resourceAcquired) Task=`175a1f4c-d1c8-4c0c-a169-4ac89b3047ea`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:25:57,348::task::990::TaskManager.Task::(_decref) Task=`175a1f4c-d1c8-4c0c-a169-4ac89b3047ea`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:25:57,349::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:25:57,349::task::1185::TaskManager.Task::(prepare) Task=`175a1f4c-d1c8-4c0c-a169-4ac89b3047ea`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:25:57,350::task::595::TaskManager.Task::(_updateState) Task=`175a1f4c-d1c8-4c0c-a169-4ac89b3047ea`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:25:57,350::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:25:57,350::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:25:57,350::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:25:57,350::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:25:57,351::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:25:57,351::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:25:57,351::task::990::TaskManager.Task::(_decref) Task=`175a1f4c-d1c8-4c0c-a169-4ac89b3047ea`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:25:57,651::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:25:58,483::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:25:58,561::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0346006 s, 29.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:26:00,574::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:00,649::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0299447 s, 34.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:26:00,669::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:26:02,662::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:03,067::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.360864 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:26:03,687::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:26:05,080::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:05,154::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.030598 s, 33.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:26:06,053::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:26:06,062::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000728732 s, 5.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:26:06,705::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:26:07,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:07,246::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358391 s, 28.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:26:07,369::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1c80df6] >Thread-13::DEBUG::2014-05-13 11:26:07,369::task::595::TaskManager.Task::(_updateState) Task=`a8f35a85-9f43-4566-a7b1-e47c6d7475a8`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:26:07,369::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:26:07,370::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:26:07,370::task::1185::TaskManager.Task::(prepare) Task=`a8f35a85-9f43-4566-a7b1-e47c6d7475a8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:26:07,370::task::595::TaskManager.Task::(_updateState) Task=`a8f35a85-9f43-4566-a7b1-e47c6d7475a8`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:26:07,370::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:26:07,370::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:26:07,371::task::990::TaskManager.Task::(_decref) Task=`a8f35a85-9f43-4566-a7b1-e47c6d7475a8`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:26:07,375::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1c80df6] >Thread-338::DEBUG::2014-05-13 11:26:07,375::task::595::TaskManager.Task::(_updateState) Task=`c422dddd-1de1-4fe8-ae3b-f2a41a5010d5`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:26:07,375::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:26:07,376::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3ce38d7e-288a-486c-b344-e709e81a211a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:26:07,376::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:26:07,376::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:26:07,376::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3ce38d7e-288a-486c-b344-e709e81a211a`::Granted request >Thread-338::DEBUG::2014-05-13 11:26:07,377::task::827::TaskManager.Task::(resourceAcquired) Task=`c422dddd-1de1-4fe8-ae3b-f2a41a5010d5`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:26:07,377::task::990::TaskManager.Task::(_decref) Task=`c422dddd-1de1-4fe8-ae3b-f2a41a5010d5`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:26:07,378::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:26:07,378::task::1185::TaskManager.Task::(prepare) Task=`c422dddd-1de1-4fe8-ae3b-f2a41a5010d5`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:26:07,378::task::595::TaskManager.Task::(_updateState) Task=`c422dddd-1de1-4fe8-ae3b-f2a41a5010d5`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:26:07,378::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:26:07,378::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:26:07,379::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:26:07,379::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:26:07,379::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:26:07,379::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:26:07,379::task::990::TaskManager.Task::(_decref) Task=`c422dddd-1de1-4fe8-ae3b-f2a41a5010d5`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:26:09,259::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:09,333::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0300908 s, 34.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:26:09,727::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:26:11,346::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:11,421::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319407 s, 32.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:26:11,908::task::595::TaskManager.Task::(_updateState) Task=`ab26ffd1-6c68-4332-9990-ade0151f78d1`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:26:11,908::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:26:11,908::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000728732', 'lastCheck': '5.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:26:11,909::task::1185::TaskManager.Task::(prepare) Task=`ab26ffd1-6c68-4332-9990-ade0151f78d1`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000728732', 'lastCheck': '5.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:26:11,909::task::595::TaskManager.Task::(_updateState) Task=`ab26ffd1-6c68-4332-9990-ade0151f78d1`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:26:11,909::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:26:11,909::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:26:11,909::task::990::TaskManager.Task::(_decref) Task=`ab26ffd1-6c68-4332-9990-ade0151f78d1`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:26:12,748::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:26:13,434::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:13,513::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0354766 s, 28.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:26:15,526::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:26:15,766::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:26:15,879::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.310015 s, 3.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:26:16,069::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:26:16,077::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000717314 s, 5.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:26:17,399::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:26:17,399::task::595::TaskManager.Task::(_updateState) Task=`b8aee497-6059-4975-be79-a5a72bdb9626`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:26:17,399::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:26:17,400::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:26:17,400::task::1185::TaskManager.Task::(prepare) Task=`b8aee497-6059-4975-be79-a5a72bdb9626`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:26:17,400::task::595::TaskManager.Task::(_updateState) Task=`b8aee497-6059-4975-be79-a5a72bdb9626`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:26:17,400::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:26:17,400::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:26:17,401::task::990::TaskManager.Task::(_decref) Task=`b8aee497-6059-4975-be79-a5a72bdb9626`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:26:17,406::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:26:17,406::task::595::TaskManager.Task::(_updateState) Task=`67cb37c8-40a7-4ec5-b96d-ad81e7fc0ac7`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:26:17,406::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:26:17,407::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`34806c03-2bc3-416f-838f-5541b1479dc2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:26:17,407::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:26:17,407::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:26:17,407::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`34806c03-2bc3-416f-838f-5541b1479dc2`::Granted request >Thread-338::DEBUG::2014-05-13 11:26:17,408::task::827::TaskManager.Task::(resourceAcquired) Task=`67cb37c8-40a7-4ec5-b96d-ad81e7fc0ac7`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:26:17,408::task::990::TaskManager.Task::(_decref) Task=`67cb37c8-40a7-4ec5-b96d-ad81e7fc0ac7`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:26:17,408::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:26:17,408::task::1185::TaskManager.Task::(prepare) Task=`67cb37c8-40a7-4ec5-b96d-ad81e7fc0ac7`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:26:17,409::task::595::TaskManager.Task::(_updateState) Task=`67cb37c8-40a7-4ec5-b96d-ad81e7fc0ac7`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:26:17,409::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:26:17,409::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:26:17,409::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:26:17,409::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:26:17,409::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:26:17,410::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:26:17,410::task::990::TaskManager.Task::(_decref) Task=`67cb37c8-40a7-4ec5-b96d-ad81e7fc0ac7`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:26:17,892::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:17,968::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319669 s, 32.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:26:18,786::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:26:19,981::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:20,057::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314657 s, 32.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:26:21,804::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:26:22,070::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:22,149::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035056 s, 29.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:26:24,162::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:24,236::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303013 s, 33.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:26:24,824::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:26:26,084::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:26:26,093::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000694844 s, 5.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:26:26,249::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:26,340::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0347383 s, 29.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:26:27,038::task::595::TaskManager.Task::(_updateState) Task=`b0afcec9-34ca-496b-a581-07de91bb9574`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:26:27,038::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:26:27,038::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000694844', 'lastCheck': '0.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:26:27,039::task::1185::TaskManager.Task::(prepare) Task=`b0afcec9-34ca-496b-a581-07de91bb9574`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000694844', 'lastCheck': '0.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:26:27,039::task::595::TaskManager.Task::(_updateState) Task=`b0afcec9-34ca-496b-a581-07de91bb9574`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:26:27,039::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:26:27,039::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:26:27,039::task::990::TaskManager.Task::(_decref) Task=`b0afcec9-34ca-496b-a581-07de91bb9574`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 11:26:27,428::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:26:27,428::task::595::TaskManager.Task::(_updateState) Task=`fda9e3f4-fdd2-4339-81ce-627a7656478e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:26:27,429::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:26:27,429::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:26:27,429::task::1185::TaskManager.Task::(prepare) Task=`fda9e3f4-fdd2-4339-81ce-627a7656478e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:26:27,430::task::595::TaskManager.Task::(_updateState) Task=`fda9e3f4-fdd2-4339-81ce-627a7656478e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:26:27,430::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:26:27,430::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:26:27,430::task::990::TaskManager.Task::(_decref) Task=`fda9e3f4-fdd2-4339-81ce-627a7656478e`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:26:27,435::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:26:27,435::task::595::TaskManager.Task::(_updateState) Task=`6b103f2e-33a5-4022-9e41-21628a546ece`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:26:27,435::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:26:27,436::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`94b6be08-3b61-4cef-bff4-6c4c680b5b79`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:26:27,436::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:26:27,436::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:26:27,436::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`94b6be08-3b61-4cef-bff4-6c4c680b5b79`::Granted request >Thread-338::DEBUG::2014-05-13 11:26:27,437::task::827::TaskManager.Task::(resourceAcquired) Task=`6b103f2e-33a5-4022-9e41-21628a546ece`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:26:27,437::task::990::TaskManager.Task::(_decref) Task=`6b103f2e-33a5-4022-9e41-21628a546ece`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:26:27,437::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:26:27,438::task::1185::TaskManager.Task::(prepare) Task=`6b103f2e-33a5-4022-9e41-21628a546ece`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:26:27,438::task::595::TaskManager.Task::(_updateState) Task=`6b103f2e-33a5-4022-9e41-21628a546ece`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:26:27,438::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:26:27,438::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:26:27,438::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:26:27,439::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:26:27,439::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:26:27,439::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:26:27,439::task::990::TaskManager.Task::(_decref) Task=`6b103f2e-33a5-4022-9e41-21628a546ece`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:26:27,844::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:26:28,353::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:28,434::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374583 s, 27.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:26:30,446::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:30,535::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0451463 s, 22.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:26:30,870::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:26:32,548::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:32,624::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317039 s, 32.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:26:33,889::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:26:34,637::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:34,718::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0354023 s, 28.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:26:36,132::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:26:36,140::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00066849 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:26:36,732::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:36,814::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0366934 s, 27.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:26:36,906::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:26:37,458::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [dc1f215] >Thread-13::DEBUG::2014-05-13 11:26:37,458::task::595::TaskManager.Task::(_updateState) Task=`0197cdcb-d559-47ed-8c93-53b5317143cf`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:26:37,459::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:26:37,459::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:26:37,459::task::1185::TaskManager.Task::(prepare) Task=`0197cdcb-d559-47ed-8c93-53b5317143cf`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:26:37,459::task::595::TaskManager.Task::(_updateState) Task=`0197cdcb-d559-47ed-8c93-53b5317143cf`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:26:37,459::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:26:37,460::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:26:37,460::task::990::TaskManager.Task::(_decref) Task=`0197cdcb-d559-47ed-8c93-53b5317143cf`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:26:37,466::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [dc1f215] >Thread-338::DEBUG::2014-05-13 11:26:37,466::task::595::TaskManager.Task::(_updateState) Task=`3c2c9329-0393-4093-8add-85f96b5acc0b`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:26:37,466::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:26:37,467::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`75aa10c6-0ec8-46db-8e7d-204423240ff6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:26:37,467::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:26:37,467::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:26:37,467::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`75aa10c6-0ec8-46db-8e7d-204423240ff6`::Granted request >Thread-338::DEBUG::2014-05-13 11:26:37,467::task::827::TaskManager.Task::(resourceAcquired) Task=`3c2c9329-0393-4093-8add-85f96b5acc0b`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:26:37,468::task::990::TaskManager.Task::(_decref) Task=`3c2c9329-0393-4093-8add-85f96b5acc0b`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:26:37,468::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:26:37,469::task::1185::TaskManager.Task::(prepare) Task=`3c2c9329-0393-4093-8add-85f96b5acc0b`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:26:37,469::task::595::TaskManager.Task::(_updateState) Task=`3c2c9329-0393-4093-8add-85f96b5acc0b`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:26:37,469::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:26:37,469::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:26:37,469::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:26:37,470::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:26:37,470::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:26:37,470::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:26:37,470::task::990::TaskManager.Task::(_decref) Task=`3c2c9329-0393-4093-8add-85f96b5acc0b`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:26:38,827::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:38,902::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0327708 s, 31.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:26:39,927::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:26:40,915::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:41,038::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0786752 s, 13.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:26:42,171::task::595::TaskManager.Task::(_updateState) Task=`df7bb064-944a-420a-80a6-278a6781ced6`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:26:42,172::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:26:42,172::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00066849', 'lastCheck': '6.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:26:42,172::task::1185::TaskManager.Task::(prepare) Task=`df7bb064-944a-420a-80a6-278a6781ced6`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00066849', 'lastCheck': '6.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:26:42,172::task::595::TaskManager.Task::(_updateState) Task=`df7bb064-944a-420a-80a6-278a6781ced6`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:26:42,172::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:26:42,172::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:26:42,173::task::990::TaskManager.Task::(_decref) Task=`df7bb064-944a-420a-80a6-278a6781ced6`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:26:42,951::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:26:43,051::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:43,148::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0510506 s, 20.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:26:45,160::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:45,235::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311412 s, 32.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:26:45,970::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:26:46,148::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:26:46,184::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.0283283 s, 145 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:26:47,248::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:47,327::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0350102 s, 29.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:26:47,491::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:26:47,492::task::595::TaskManager.Task::(_updateState) Task=`9a8515af-8b56-4000-93ed-dde2b2b207c5`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:26:47,492::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:26:47,492::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:26:47,492::task::1185::TaskManager.Task::(prepare) Task=`9a8515af-8b56-4000-93ed-dde2b2b207c5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:26:47,493::task::595::TaskManager.Task::(_updateState) Task=`9a8515af-8b56-4000-93ed-dde2b2b207c5`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:26:47,493::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:26:47,493::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:26:47,493::task::990::TaskManager.Task::(_decref) Task=`9a8515af-8b56-4000-93ed-dde2b2b207c5`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:26:47,498::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:26:47,498::task::595::TaskManager.Task::(_updateState) Task=`344fea82-5434-46b9-94f0-5220f9fab55b`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:26:47,498::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:26:47,499::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a42f998b-7016-4e0e-a4d9-d968b6bbca2e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:26:47,499::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:26:47,499::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:26:47,499::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a42f998b-7016-4e0e-a4d9-d968b6bbca2e`::Granted request >Thread-338::DEBUG::2014-05-13 11:26:47,500::task::827::TaskManager.Task::(resourceAcquired) Task=`344fea82-5434-46b9-94f0-5220f9fab55b`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:26:47,500::task::990::TaskManager.Task::(_decref) Task=`344fea82-5434-46b9-94f0-5220f9fab55b`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:26:47,501::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:26:47,501::task::1185::TaskManager.Task::(prepare) Task=`344fea82-5434-46b9-94f0-5220f9fab55b`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:26:47,501::task::595::TaskManager.Task::(_updateState) Task=`344fea82-5434-46b9-94f0-5220f9fab55b`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:26:47,501::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:26:47,501::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:26:47,502::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:26:47,502::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:26:47,502::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:26:47,502::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:26:47,502::task::990::TaskManager.Task::(_decref) Task=`344fea82-5434-46b9-94f0-5220f9fab55b`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:26:48,985::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:26:49,340::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:49,420::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0351169 s, 29.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:26:51,433::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:51,509::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314828 s, 32.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:26:52,004::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:26:53,522::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:53,595::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0302959 s, 33.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:26:55,022::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:26:55,608::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:55,687::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0349951 s, 29.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:26:56,190::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:26:56,203::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00449384 s, 911 kB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:26:57,308::task::595::TaskManager.Task::(_updateState) Task=`49d1910d-fdab-4a58-b0da-a37040c36c7a`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:26:57,308::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:26:57,309::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00449384', 'lastCheck': '1.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:26:57,309::task::1185::TaskManager.Task::(prepare) Task=`49d1910d-fdab-4a58-b0da-a37040c36c7a`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00449384', 'lastCheck': '1.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:26:57,309::task::595::TaskManager.Task::(_updateState) Task=`49d1910d-fdab-4a58-b0da-a37040c36c7a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:26:57,309::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:26:57,309::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:26:57,309::task::990::TaskManager.Task::(_decref) Task=`49d1910d-fdab-4a58-b0da-a37040c36c7a`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 11:26:57,524::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:26:57,524::task::595::TaskManager.Task::(_updateState) Task=`b430e131-6f0f-4d51-bd58-677e4d4fff14`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:26:57,524::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:26:57,524::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:26:57,525::task::1185::TaskManager.Task::(prepare) Task=`b430e131-6f0f-4d51-bd58-677e4d4fff14`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:26:57,525::task::595::TaskManager.Task::(_updateState) Task=`b430e131-6f0f-4d51-bd58-677e4d4fff14`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:26:57,525::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:26:57,525::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:26:57,525::task::990::TaskManager.Task::(_decref) Task=`b430e131-6f0f-4d51-bd58-677e4d4fff14`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:26:57,530::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:26:57,530::task::595::TaskManager.Task::(_updateState) Task=`bcc5bca9-9325-49ea-9e16-e1058b0c349c`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:26:57,530::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:26:57,531::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c613bfe0-86f1-45bc-b2cd-1822ffdcd7c5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:26:57,531::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:26:57,531::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:26:57,531::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c613bfe0-86f1-45bc-b2cd-1822ffdcd7c5`::Granted request >Thread-338::DEBUG::2014-05-13 11:26:57,532::task::827::TaskManager.Task::(resourceAcquired) Task=`bcc5bca9-9325-49ea-9e16-e1058b0c349c`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:26:57,532::task::990::TaskManager.Task::(_decref) Task=`bcc5bca9-9325-49ea-9e16-e1058b0c349c`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:26:57,533::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:26:57,533::task::1185::TaskManager.Task::(prepare) Task=`bcc5bca9-9325-49ea-9e16-e1058b0c349c`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:26:57,533::task::595::TaskManager.Task::(_updateState) Task=`bcc5bca9-9325-49ea-9e16-e1058b0c349c`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:26:57,533::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:26:57,533::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:26:57,534::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:26:57,534::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:26:57,534::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:26:57,534::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:26:57,534::task::990::TaskManager.Task::(_decref) Task=`bcc5bca9-9325-49ea-9e16-e1058b0c349c`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:26:57,700::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:57,793::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0501812 s, 20.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:26:58,054::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:26:59,806::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:26:59,880::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0308716 s, 33.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:27:01,074::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:27:01,893::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:01,971::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0344545 s, 29.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:27:03,984::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:04,064::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0366856 s, 27.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:27:04,092::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:27:06,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:06,152::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314747 s, 32.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:27:06,209::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:27:06,229::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.0122661 s, 334 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:27:07,110::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:27:07,555::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:27:07,556::task::595::TaskManager.Task::(_updateState) Task=`56bfa461-ca3f-4f53-ad04-d1d908a51e67`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:27:07,556::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:27:07,556::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:27:07,556::task::1185::TaskManager.Task::(prepare) Task=`56bfa461-ca3f-4f53-ad04-d1d908a51e67`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:27:07,557::task::595::TaskManager.Task::(_updateState) Task=`56bfa461-ca3f-4f53-ad04-d1d908a51e67`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:27:07,557::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:27:07,557::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:27:07,557::task::990::TaskManager.Task::(_decref) Task=`56bfa461-ca3f-4f53-ad04-d1d908a51e67`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:27:07,562::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:27:07,562::task::595::TaskManager.Task::(_updateState) Task=`f6cea137-6c1b-42a3-b511-88a91e0d458f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:27:07,562::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:27:07,563::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`20af4e1a-e3a8-4b0b-bb33-d1d7d11245b9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:27:07,563::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:27:07,563::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:27:07,563::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`20af4e1a-e3a8-4b0b-bb33-d1d7d11245b9`::Granted request >Thread-338::DEBUG::2014-05-13 11:27:07,564::task::827::TaskManager.Task::(resourceAcquired) Task=`f6cea137-6c1b-42a3-b511-88a91e0d458f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:27:07,564::task::990::TaskManager.Task::(_decref) Task=`f6cea137-6c1b-42a3-b511-88a91e0d458f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:27:07,565::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:27:07,565::task::1185::TaskManager.Task::(prepare) Task=`f6cea137-6c1b-42a3-b511-88a91e0d458f`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:27:07,565::task::595::TaskManager.Task::(_updateState) Task=`f6cea137-6c1b-42a3-b511-88a91e0d458f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:27:07,565::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:27:07,565::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:27:07,566::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:27:07,566::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:27:07,566::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:27:07,566::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:27:07,566::task::990::TaskManager.Task::(_decref) Task=`f6cea137-6c1b-42a3-b511-88a91e0d458f`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:27:08,165::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:08,238::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0301497 s, 34.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:27:10,134::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:27:10,251::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:10,330::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353194 s, 29.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:27:12,343::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 11:27:12,449::task::595::TaskManager.Task::(_updateState) Task=`e88808b5-5f73-492b-9179-a54ca6f2d2dc`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:27:12,449::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:27:12,449::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0122661', 'lastCheck': '6.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:27:12,449::task::1185::TaskManager.Task::(prepare) Task=`e88808b5-5f73-492b-9179-a54ca6f2d2dc`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0122661', 'lastCheck': '6.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:27:12,449::task::595::TaskManager.Task::(_updateState) Task=`e88808b5-5f73-492b-9179-a54ca6f2d2dc`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:27:12,450::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:27:12,450::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:27:12,450::task::990::TaskManager.Task::(_decref) Task=`e88808b5-5f73-492b-9179-a54ca6f2d2dc`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:27:12,757::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.372023 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:27:13,153::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:27:14,770::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:14,845::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323973 s, 31.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:27:16,171::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:27:16,236::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:27:16,245::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000648256 s, 6.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:27:16,858::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:16,938::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365653 s, 28.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:27:17,587::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6019f20] >Thread-13::DEBUG::2014-05-13 11:27:17,587::task::595::TaskManager.Task::(_updateState) Task=`c725754d-395d-4b61-bef1-271937d385e5`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:27:17,588::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:27:17,588::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:27:17,588::task::1185::TaskManager.Task::(prepare) Task=`c725754d-395d-4b61-bef1-271937d385e5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:27:17,588::task::595::TaskManager.Task::(_updateState) Task=`c725754d-395d-4b61-bef1-271937d385e5`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:27:17,588::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:27:17,589::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:27:17,589::task::990::TaskManager.Task::(_decref) Task=`c725754d-395d-4b61-bef1-271937d385e5`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:27:17,593::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6019f20] >Thread-338::DEBUG::2014-05-13 11:27:17,593::task::595::TaskManager.Task::(_updateState) Task=`95830b7b-4bb7-4164-824f-ca0848f581ac`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:27:17,593::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:27:17,594::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f9557071-7138-4a8a-84c8-cb35180a21a9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:27:17,594::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:27:17,594::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:27:17,594::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f9557071-7138-4a8a-84c8-cb35180a21a9`::Granted request >Thread-338::DEBUG::2014-05-13 11:27:17,595::task::827::TaskManager.Task::(resourceAcquired) Task=`95830b7b-4bb7-4164-824f-ca0848f581ac`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:27:17,595::task::990::TaskManager.Task::(_decref) Task=`95830b7b-4bb7-4164-824f-ca0848f581ac`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:27:17,596::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:27:17,596::task::1185::TaskManager.Task::(prepare) Task=`95830b7b-4bb7-4164-824f-ca0848f581ac`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:27:17,596::task::595::TaskManager.Task::(_updateState) Task=`95830b7b-4bb7-4164-824f-ca0848f581ac`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:27:17,596::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:27:17,596::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:27:17,597::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:27:17,597::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:27:17,597::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:27:17,597::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:27:17,597::task::990::TaskManager.Task::(_decref) Task=`95830b7b-4bb7-4164-824f-ca0848f581ac`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:27:18,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:19,030::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0354344 s, 28.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:27:19,190::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:27:21,043::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:21,275::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307605 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:27:22,210::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:27:23,288::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:23,691::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.35966 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:27:25,230::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:27:25,704::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:25,782::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0350604 s, 29.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:27:26,252::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:27:26,260::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000589431 s, 6.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:27:27,578::task::595::TaskManager.Task::(_updateState) Task=`75f89f88-f56b-438b-a30c-56266c4eb967`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:27:27,579::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:27:27,579::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000589431', 'lastCheck': '1.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:27:27,579::task::1185::TaskManager.Task::(prepare) Task=`75f89f88-f56b-438b-a30c-56266c4eb967`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000589431', 'lastCheck': '1.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:27:27,579::task::595::TaskManager.Task::(_updateState) Task=`75f89f88-f56b-438b-a30c-56266c4eb967`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:27:27,579::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:27:27,579::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:27:27,580::task::990::TaskManager.Task::(_decref) Task=`75f89f88-f56b-438b-a30c-56266c4eb967`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 11:27:27,617::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:27:27,618::task::595::TaskManager.Task::(_updateState) Task=`ed26354c-06d0-450e-b2bc-12ec07aa0774`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:27:27,618::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:27:27,618::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:27:27,618::task::1185::TaskManager.Task::(prepare) Task=`ed26354c-06d0-450e-b2bc-12ec07aa0774`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:27:27,619::task::595::TaskManager.Task::(_updateState) Task=`ed26354c-06d0-450e-b2bc-12ec07aa0774`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:27:27,619::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:27:27,619::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:27:27,619::task::990::TaskManager.Task::(_decref) Task=`ed26354c-06d0-450e-b2bc-12ec07aa0774`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:27:27,624::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:27:27,624::task::595::TaskManager.Task::(_updateState) Task=`0b64634c-e96e-4501-afc0-60815e087cc4`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:27:27,625::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:27:27,625::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1930b539-0e96-4c9e-b86f-a300ba96e430`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:27:27,625::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:27:27,625::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:27:27,626::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1930b539-0e96-4c9e-b86f-a300ba96e430`::Granted request >Thread-338::DEBUG::2014-05-13 11:27:27,626::task::827::TaskManager.Task::(resourceAcquired) Task=`0b64634c-e96e-4501-afc0-60815e087cc4`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:27:27,626::task::990::TaskManager.Task::(_decref) Task=`0b64634c-e96e-4501-afc0-60815e087cc4`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:27:27,627::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:27:27,627::task::1185::TaskManager.Task::(prepare) Task=`0b64634c-e96e-4501-afc0-60815e087cc4`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:27:27,627::task::595::TaskManager.Task::(_updateState) Task=`0b64634c-e96e-4501-afc0-60815e087cc4`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:27:27,628::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:27:27,628::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:27:27,628::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:27:27,628::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:27:27,628::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:27:27,629::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:27:27,629::task::990::TaskManager.Task::(_decref) Task=`0b64634c-e96e-4501-afc0-60815e087cc4`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:27:27,795::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:27,875::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0375289 s, 27.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:27:28,252::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:27:29,888::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:29,963::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314879 s, 32.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:27:31,270::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:27:31,977::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:32,049::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0351159 s, 29.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:27:34,057::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:34,137::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373465 s, 27.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:27:34,289::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:27:36,150::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:36,224::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.029943 s, 34.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:27:36,267::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:27:36,276::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000680576 s, 6.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:27:37,309::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:27:37,656::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1ea398d3] >Thread-13::DEBUG::2014-05-13 11:27:37,656::task::595::TaskManager.Task::(_updateState) Task=`dfe54728-4785-47e9-83b0-7d1aec838b9e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:27:37,657::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:27:37,657::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:27:37,657::task::1185::TaskManager.Task::(prepare) Task=`dfe54728-4785-47e9-83b0-7d1aec838b9e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:27:37,657::task::595::TaskManager.Task::(_updateState) Task=`dfe54728-4785-47e9-83b0-7d1aec838b9e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:27:37,658::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:27:37,658::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:27:37,658::task::990::TaskManager.Task::(_decref) Task=`dfe54728-4785-47e9-83b0-7d1aec838b9e`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:27:38,120::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1ea398d3] >Thread-338::DEBUG::2014-05-13 11:27:38,121::task::595::TaskManager.Task::(_updateState) Task=`8526beb5-f1c7-47e2-b436-14d2add80718`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:27:38,121::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:27:38,121::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`57f3cc06-bd36-41af-8a7c-c497b3a834a3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:27:38,121::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:27:38,122::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:27:38,122::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`57f3cc06-bd36-41af-8a7c-c497b3a834a3`::Granted request >Thread-338::DEBUG::2014-05-13 11:27:38,122::task::827::TaskManager.Task::(resourceAcquired) Task=`8526beb5-f1c7-47e2-b436-14d2add80718`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:27:38,122::task::990::TaskManager.Task::(_decref) Task=`8526beb5-f1c7-47e2-b436-14d2add80718`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:27:38,123::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:27:38,123::task::1185::TaskManager.Task::(prepare) Task=`8526beb5-f1c7-47e2-b436-14d2add80718`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:27:38,123::task::595::TaskManager.Task::(_updateState) Task=`8526beb5-f1c7-47e2-b436-14d2add80718`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:27:38,123::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:27:38,124::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:27:38,124::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:27:38,124::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:27:38,124::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:27:38,124::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:27:38,124::task::990::TaskManager.Task::(_decref) Task=`8526beb5-f1c7-47e2-b436-14d2add80718`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:27:38,236::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:38,322::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0430903 s, 23.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:27:40,328::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:27:40,335::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:40,415::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363085 s, 28.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:27:42,428::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:42,507::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0372445 s, 27.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:27:42,718::task::595::TaskManager.Task::(_updateState) Task=`35213ba1-53b3-4f5d-bf3e-c7f565b0947b`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:27:42,718::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:27:42,718::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000680576', 'lastCheck': '6.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:27:42,719::task::1185::TaskManager.Task::(prepare) Task=`35213ba1-53b3-4f5d-bf3e-c7f565b0947b`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000680576', 'lastCheck': '6.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:27:42,719::task::595::TaskManager.Task::(_updateState) Task=`35213ba1-53b3-4f5d-bf3e-c7f565b0947b`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:27:42,719::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:27:42,719::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:27:42,719::task::990::TaskManager.Task::(_decref) Task=`35213ba1-53b3-4f5d-bf3e-c7f565b0947b`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:27:43,348::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:27:44,531::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:44,607::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0330034 s, 31.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:27:46,283::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:27:46,292::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00134911 s, 3.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:27:46,367::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:27:46,619::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:46,698::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355387 s, 28.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:27:48,144::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:27:48,183::task::595::TaskManager.Task::(_updateState) Task=`f3385a75-214f-44bc-863e-24abcfaadc2f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:27:48,183::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:27:48,183::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:27:48,183::task::1185::TaskManager.Task::(prepare) Task=`f3385a75-214f-44bc-863e-24abcfaadc2f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:27:48,184::task::595::TaskManager.Task::(_updateState) Task=`f3385a75-214f-44bc-863e-24abcfaadc2f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:27:48,184::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:27:48,184::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:27:48,184::task::990::TaskManager.Task::(_decref) Task=`f3385a75-214f-44bc-863e-24abcfaadc2f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:27:48,189::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:27:48,189::task::595::TaskManager.Task::(_updateState) Task=`e047ba8e-be6c-4c62-9a60-7daa6f174eb8`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:27:48,190::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:27:48,190::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9a6cc169-7fd1-4057-bf41-042949d93a0f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:27:48,190::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:27:48,191::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:27:48,191::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9a6cc169-7fd1-4057-bf41-042949d93a0f`::Granted request >Thread-338::DEBUG::2014-05-13 11:27:48,191::task::827::TaskManager.Task::(resourceAcquired) Task=`e047ba8e-be6c-4c62-9a60-7daa6f174eb8`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:27:48,191::task::990::TaskManager.Task::(_decref) Task=`e047ba8e-be6c-4c62-9a60-7daa6f174eb8`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:27:48,192::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:27:48,192::task::1185::TaskManager.Task::(prepare) Task=`e047ba8e-be6c-4c62-9a60-7daa6f174eb8`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:27:48,192::task::595::TaskManager.Task::(_updateState) Task=`e047ba8e-be6c-4c62-9a60-7daa6f174eb8`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:27:48,193::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:27:48,193::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:27:48,193::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:27:48,193::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:27:48,193::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:27:48,194::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:27:48,194::task::990::TaskManager.Task::(_decref) Task=`e047ba8e-be6c-4c62-9a60-7daa6f174eb8`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:27:48,711::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:48,788::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0345063 s, 29.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:27:49,386::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:27:50,802::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:50,875::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.029661 s, 34.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:27:52,405::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:27:52,888::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:53,253::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.322023 s, 3.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:27:55,265::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:55,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0347962 s, 29.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:27:55,427::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:27:56,299::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:27:56,307::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000636191 s, 6.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:27:57,357::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:57,438::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361807 s, 28.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:27:57,849::task::595::TaskManager.Task::(_updateState) Task=`691a70d4-71dc-4039-bfa3-fe4adbc1eca5`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:27:57,850::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:27:57,851::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000636191', 'lastCheck': '1.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:27:57,851::task::1185::TaskManager.Task::(prepare) Task=`691a70d4-71dc-4039-bfa3-fe4adbc1eca5`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000636191', 'lastCheck': '1.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:27:57,851::task::595::TaskManager.Task::(_updateState) Task=`691a70d4-71dc-4039-bfa3-fe4adbc1eca5`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:27:57,851::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:27:57,851::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:27:57,852::task::990::TaskManager.Task::(_decref) Task=`691a70d4-71dc-4039-bfa3-fe4adbc1eca5`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 11:27:58,212::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5affc0d0] >Thread-13::DEBUG::2014-05-13 11:27:58,212::task::595::TaskManager.Task::(_updateState) Task=`8c54e8ea-c565-4b94-9bfa-c75878c9ac6f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:27:58,212::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:27:58,213::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:27:58,213::task::1185::TaskManager.Task::(prepare) Task=`8c54e8ea-c565-4b94-9bfa-c75878c9ac6f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:27:58,213::task::595::TaskManager.Task::(_updateState) Task=`8c54e8ea-c565-4b94-9bfa-c75878c9ac6f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:27:58,213::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:27:58,213::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:27:58,213::task::990::TaskManager.Task::(_decref) Task=`8c54e8ea-c565-4b94-9bfa-c75878c9ac6f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:27:58,219::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5affc0d0] >Thread-338::DEBUG::2014-05-13 11:27:58,220::task::595::TaskManager.Task::(_updateState) Task=`52a55e43-78cd-4573-a741-3bf7024a4f2e`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:27:58,220::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:27:58,220::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`13306067-8138-4b2b-a237-852d3148775e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:27:58,220::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:27:58,221::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:27:58,221::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`13306067-8138-4b2b-a237-852d3148775e`::Granted request >Thread-338::DEBUG::2014-05-13 11:27:58,221::task::827::TaskManager.Task::(resourceAcquired) Task=`52a55e43-78cd-4573-a741-3bf7024a4f2e`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:27:58,221::task::990::TaskManager.Task::(_decref) Task=`52a55e43-78cd-4573-a741-3bf7024a4f2e`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:27:58,222::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:27:58,222::task::1185::TaskManager.Task::(prepare) Task=`52a55e43-78cd-4573-a741-3bf7024a4f2e`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:27:58,222::task::595::TaskManager.Task::(_updateState) Task=`52a55e43-78cd-4573-a741-3bf7024a4f2e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:27:58,223::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:27:58,223::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:27:58,223::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:27:58,223::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:27:58,223::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:27:58,224::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:27:58,224::task::990::TaskManager.Task::(_decref) Task=`52a55e43-78cd-4573-a741-3bf7024a4f2e`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:27:58,446::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:27:59,451::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:27:59,525::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305685 s, 33.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:28:01,467::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:28:01,538::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:01,615::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0347086 s, 29.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:28:03,628::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:03,708::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035183 s, 29.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:28:04,485::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:28:05,722::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:05,809::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0438103 s, 23.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:28:06,315::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:28:06,323::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000652832 s, 6.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:28:07,505::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:28:07,822::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:07,898::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.034891 s, 29.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:28:08,244::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4dbe430a] >Thread-13::DEBUG::2014-05-13 11:28:08,244::task::595::TaskManager.Task::(_updateState) Task=`a0a1e345-00a8-4c1f-8ab1-ca003b598f39`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:28:08,245::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:28:08,245::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:28:08,245::task::1185::TaskManager.Task::(prepare) Task=`a0a1e345-00a8-4c1f-8ab1-ca003b598f39`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:28:08,245::task::595::TaskManager.Task::(_updateState) Task=`a0a1e345-00a8-4c1f-8ab1-ca003b598f39`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:28:08,246::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:28:08,246::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:28:08,246::task::990::TaskManager.Task::(_decref) Task=`a0a1e345-00a8-4c1f-8ab1-ca003b598f39`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:28:08,251::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4dbe430a] >Thread-338::DEBUG::2014-05-13 11:28:08,251::task::595::TaskManager.Task::(_updateState) Task=`9bd391a2-38a5-474f-94f2-24ecf69fc04f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:28:08,251::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:28:08,252::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f0c10eb8-3e46-4316-9ce7-2d56fcebfdcf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:28:08,252::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:28:08,252::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:28:08,252::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f0c10eb8-3e46-4316-9ce7-2d56fcebfdcf`::Granted request >Thread-338::DEBUG::2014-05-13 11:28:08,253::task::827::TaskManager.Task::(resourceAcquired) Task=`9bd391a2-38a5-474f-94f2-24ecf69fc04f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:28:08,253::task::990::TaskManager.Task::(_decref) Task=`9bd391a2-38a5-474f-94f2-24ecf69fc04f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:28:08,254::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:28:08,254::task::1185::TaskManager.Task::(prepare) Task=`9bd391a2-38a5-474f-94f2-24ecf69fc04f`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:28:08,254::task::595::TaskManager.Task::(_updateState) Task=`9bd391a2-38a5-474f-94f2-24ecf69fc04f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:28:08,254::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:28:08,255::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:28:08,255::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:28:08,255::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:28:08,255::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:28:08,255::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:28:08,256::task::990::TaskManager.Task::(_decref) Task=`9bd391a2-38a5-474f-94f2-24ecf69fc04f`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:28:09,911::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:09,988::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0294967 s, 34.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:28:10,525::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:28:12,001::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:12,075::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0300561 s, 34.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:28:12,988::task::595::TaskManager.Task::(_updateState) Task=`d89629f0-4161-4b7c-a2a5-8caf0ca20de2`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:28:12,988::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:28:12,989::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000652832', 'lastCheck': '6.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:28:12,989::task::1185::TaskManager.Task::(prepare) Task=`d89629f0-4161-4b7c-a2a5-8caf0ca20de2`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000652832', 'lastCheck': '6.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:28:12,989::task::595::TaskManager.Task::(_updateState) Task=`d89629f0-4161-4b7c-a2a5-8caf0ca20de2`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:28:12,989::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:28:12,989::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:28:12,989::task::990::TaskManager.Task::(_decref) Task=`d89629f0-4161-4b7c-a2a5-8caf0ca20de2`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:28:13,543::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:28:14,088::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:14,161::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0299627 s, 34.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:28:16,174::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:16,257::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381346 s, 26.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:28:16,350::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:28:16,359::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000643766 s, 6.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:28:16,562::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:28:18,270::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 11:28:18,280::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:28:18,281::task::595::TaskManager.Task::(_updateState) Task=`f956d699-c107-4e9c-8d10-8c131b170142`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:28:18,281::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:28:18,281::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:28:18,282::task::1185::TaskManager.Task::(prepare) Task=`f956d699-c107-4e9c-8d10-8c131b170142`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:28:18,282::task::595::TaskManager.Task::(_updateState) Task=`f956d699-c107-4e9c-8d10-8c131b170142`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:28:18,282::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:28:18,282::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:28:18,283::task::990::TaskManager.Task::(_decref) Task=`f956d699-c107-4e9c-8d10-8c131b170142`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:28:18,287::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:28:18,288::task::595::TaskManager.Task::(_updateState) Task=`94b39ee4-3f3a-4efb-bd61-38ad8b3661f2`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:28:18,288::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:28:18,289::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`75d24c86-4595-4b40-927a-395f996f2718`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:28:18,289::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:28:18,289::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:28:18,289::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`75d24c86-4595-4b40-927a-395f996f2718`::Granted request >Thread-338::DEBUG::2014-05-13 11:28:18,290::task::827::TaskManager.Task::(resourceAcquired) Task=`94b39ee4-3f3a-4efb-bd61-38ad8b3661f2`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:28:18,290::task::990::TaskManager.Task::(_decref) Task=`94b39ee4-3f3a-4efb-bd61-38ad8b3661f2`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:28:18,291::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:28:18,291::task::1185::TaskManager.Task::(prepare) Task=`94b39ee4-3f3a-4efb-bd61-38ad8b3661f2`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:28:18,292::task::595::TaskManager.Task::(_updateState) Task=`94b39ee4-3f3a-4efb-bd61-38ad8b3661f2`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:28:18,292::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:28:18,292::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:28:18,292::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:28:18,293::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:28:18,293::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:28:18,293::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:28:18,293::task::990::TaskManager.Task::(_decref) Task=`94b39ee4-3f3a-4efb-bd61-38ad8b3661f2`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:28:18,362::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0471548 s, 21.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:28:19,577::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:28:20,375::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:20,449::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306184 s, 33.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:28:22,461::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:22,539::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0343546 s, 29.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:28:22,595::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:28:24,552::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:24,640::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.030558 s, 33.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:28:25,619::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:28:26,365::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:28:26,374::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000652788 s, 6.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:28:26,654::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:26,732::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0348977 s, 29.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:28:28,133::task::595::TaskManager.Task::(_updateState) Task=`d9581bb7-c13f-410c-b88e-988e98e102e7`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:28:28,134::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:28:28,134::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000652788', 'lastCheck': '1.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:28:28,134::task::1185::TaskManager.Task::(prepare) Task=`d9581bb7-c13f-410c-b88e-988e98e102e7`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000652788', 'lastCheck': '1.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:28:28,134::task::595::TaskManager.Task::(_updateState) Task=`d9581bb7-c13f-410c-b88e-988e98e102e7`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:28:28,135::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:28:28,135::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:28:28,135::task::990::TaskManager.Task::(_decref) Task=`d9581bb7-c13f-410c-b88e-988e98e102e7`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 11:28:28,312::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6602059] >Thread-13::DEBUG::2014-05-13 11:28:28,313::task::595::TaskManager.Task::(_updateState) Task=`5f3e4a93-677a-4fe8-b5b4-edb26c4a0907`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:28:28,313::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:28:28,313::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:28:28,313::task::1185::TaskManager.Task::(prepare) Task=`5f3e4a93-677a-4fe8-b5b4-edb26c4a0907`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:28:28,313::task::595::TaskManager.Task::(_updateState) Task=`5f3e4a93-677a-4fe8-b5b4-edb26c4a0907`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:28:28,314::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:28:28,314::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:28:28,314::task::990::TaskManager.Task::(_decref) Task=`5f3e4a93-677a-4fe8-b5b4-edb26c4a0907`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:28:28,318::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6602059] >Thread-338::DEBUG::2014-05-13 11:28:28,318::task::595::TaskManager.Task::(_updateState) Task=`7d569f4f-87a7-4bf3-bb59-117708cfa552`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:28:28,319::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:28:28,319::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`264b597a-1cc4-4e11-8a43-49058b2e4414`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:28:28,319::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:28:28,320::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:28:28,320::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`264b597a-1cc4-4e11-8a43-49058b2e4414`::Granted request >Thread-338::DEBUG::2014-05-13 11:28:28,320::task::827::TaskManager.Task::(resourceAcquired) Task=`7d569f4f-87a7-4bf3-bb59-117708cfa552`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:28:28,320::task::990::TaskManager.Task::(_decref) Task=`7d569f4f-87a7-4bf3-bb59-117708cfa552`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:28:28,321::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:28:28,321::task::1185::TaskManager.Task::(prepare) Task=`7d569f4f-87a7-4bf3-bb59-117708cfa552`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:28:28,321::task::595::TaskManager.Task::(_updateState) Task=`7d569f4f-87a7-4bf3-bb59-117708cfa552`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:28:28,322::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:28:28,322::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:28:28,322::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:28:28,322::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:28:28,322::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:28:28,322::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:28:28,323::task::990::TaskManager.Task::(_decref) Task=`7d569f4f-87a7-4bf3-bb59-117708cfa552`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:28:28,640::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:28:28,745::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:28,825::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356759 s, 28.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:28:30,838::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:30,910::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0297452 s, 34.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:28:31,659::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:28:32,923::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:33,016::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0499422 s, 20.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:28:34,678::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:28:35,029::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:35,105::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0378285 s, 27.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:28:36,380::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:28:36,389::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000666299 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:28:37,113::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:37,193::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362124 s, 28.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:28:37,699::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:28:38,341::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5fad7764] >Thread-13::DEBUG::2014-05-13 11:28:38,342::task::595::TaskManager.Task::(_updateState) Task=`8c8e62c9-70b3-4423-a5aa-3a8c49acd123`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:28:38,342::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:28:38,342::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:28:38,343::task::1185::TaskManager.Task::(prepare) Task=`8c8e62c9-70b3-4423-a5aa-3a8c49acd123`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:28:38,343::task::595::TaskManager.Task::(_updateState) Task=`8c8e62c9-70b3-4423-a5aa-3a8c49acd123`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:28:38,343::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:28:38,343::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:28:38,343::task::990::TaskManager.Task::(_decref) Task=`8c8e62c9-70b3-4423-a5aa-3a8c49acd123`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:28:38,348::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5fad7764] >Thread-338::DEBUG::2014-05-13 11:28:38,349::task::595::TaskManager.Task::(_updateState) Task=`0366439b-9d4b-4a7f-87f0-09479698e02a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:28:38,349::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:28:38,349::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`53f43df9-754e-4f1b-a962-0095a1f5d45a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:28:38,349::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:28:38,350::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:28:38,350::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`53f43df9-754e-4f1b-a962-0095a1f5d45a`::Granted request >Thread-338::DEBUG::2014-05-13 11:28:38,350::task::827::TaskManager.Task::(resourceAcquired) Task=`0366439b-9d4b-4a7f-87f0-09479698e02a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:28:38,350::task::990::TaskManager.Task::(_decref) Task=`0366439b-9d4b-4a7f-87f0-09479698e02a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:28:38,351::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:28:38,351::task::1185::TaskManager.Task::(prepare) Task=`0366439b-9d4b-4a7f-87f0-09479698e02a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:28:38,351::task::595::TaskManager.Task::(_updateState) Task=`0366439b-9d4b-4a7f-87f0-09479698e02a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:28:38,351::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:28:38,352::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:28:38,352::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:28:38,352::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:28:38,352::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:28:38,352::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:28:38,352::task::990::TaskManager.Task::(_decref) Task=`0366439b-9d4b-4a7f-87f0-09479698e02a`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:28:39,206::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:39,280::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0298969 s, 34.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:28:40,717::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:28:41,293::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:41,366::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0298514 s, 34.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:28:43,274::task::595::TaskManager.Task::(_updateState) Task=`102dc90f-536f-409d-a2f3-9cfaeb16f1fb`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:28:43,274::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:28:43,274::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000666299', 'lastCheck': '6.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:28:43,274::task::1185::TaskManager.Task::(prepare) Task=`102dc90f-536f-409d-a2f3-9cfaeb16f1fb`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000666299', 'lastCheck': '6.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:28:43,275::task::595::TaskManager.Task::(_updateState) Task=`102dc90f-536f-409d-a2f3-9cfaeb16f1fb`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:28:43,275::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:28:43,275::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:28:43,275::task::990::TaskManager.Task::(_decref) Task=`102dc90f-536f-409d-a2f3-9cfaeb16f1fb`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:28:43,379::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:43,458::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365548 s, 28.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:28:43,739::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:28:45,471::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:46,088::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.572504 s, 1.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:28:46,414::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:28:46,422::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000761772 s, 5.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:28:46,757::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:28:48,101::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:48,180::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352435 s, 29.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:28:48,374::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5eff0b16] >Thread-13::DEBUG::2014-05-13 11:28:48,387::task::595::TaskManager.Task::(_updateState) Task=`c9c0b0b0-c90f-4512-a65b-221560829163`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:28:48,387::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:28:48,388::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:28:48,388::task::1185::TaskManager.Task::(prepare) Task=`c9c0b0b0-c90f-4512-a65b-221560829163`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:28:48,388::task::595::TaskManager.Task::(_updateState) Task=`c9c0b0b0-c90f-4512-a65b-221560829163`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:28:48,388::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:28:48,389::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:28:48,389::task::990::TaskManager.Task::(_decref) Task=`c9c0b0b0-c90f-4512-a65b-221560829163`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:28:48,395::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5eff0b16] >Thread-338::DEBUG::2014-05-13 11:28:48,395::task::595::TaskManager.Task::(_updateState) Task=`5afa6455-763b-44c2-a567-2bc4a3aa90da`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:28:48,396::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:28:48,396::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`672a7740-33af-4050-8428-1e680c59ccf5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:28:48,396::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:28:48,397::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:28:48,397::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`672a7740-33af-4050-8428-1e680c59ccf5`::Granted request >Thread-338::DEBUG::2014-05-13 11:28:48,397::task::827::TaskManager.Task::(resourceAcquired) Task=`5afa6455-763b-44c2-a567-2bc4a3aa90da`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:28:48,397::task::990::TaskManager.Task::(_decref) Task=`5afa6455-763b-44c2-a567-2bc4a3aa90da`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:28:48,398::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:28:48,398::task::1185::TaskManager.Task::(prepare) Task=`5afa6455-763b-44c2-a567-2bc4a3aa90da`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:28:48,398::task::595::TaskManager.Task::(_updateState) Task=`5afa6455-763b-44c2-a567-2bc4a3aa90da`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:28:48,399::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:28:48,399::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:28:48,399::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:28:48,399::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:28:48,399::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:28:48,399::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:28:48,400::task::990::TaskManager.Task::(_decref) Task=`5afa6455-763b-44c2-a567-2bc4a3aa90da`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:28:49,775::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:28:50,193::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:50,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306007 s, 33.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:28:52,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:52,357::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0348181 s, 29.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:28:52,794::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:28:54,370::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:54,445::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0308148 s, 33.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:28:55,823::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:28:56,429::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:28:56,437::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000698492 s, 5.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:28:56,458::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:56,536::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352561 s, 29.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:28:58,423::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:28:58,424::task::595::TaskManager.Task::(_updateState) Task=`52d26280-1a02-4437-b9fa-d27ed7792f4f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:28:58,424::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:28:58,425::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:28:58,427::task::1185::TaskManager.Task::(prepare) Task=`52d26280-1a02-4437-b9fa-d27ed7792f4f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:28:58,428::task::595::TaskManager.Task::(_updateState) Task=`52d26280-1a02-4437-b9fa-d27ed7792f4f`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:28:58,430::task::595::TaskManager.Task::(_updateState) Task=`dbf36a9d-b9a1-4745-8681-f91160224cb6`::moving from state init -> state preparing >Thread-13::DEBUG::2014-05-13 11:28:58,430::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::INFO::2014-05-13 11:28:58,431::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::DEBUG::2014-05-13 11:28:58,432::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::INFO::2014-05-13 11:28:58,433::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000698492', 'lastCheck': '2.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:28:58,434::task::990::TaskManager.Task::(_decref) Task=`52d26280-1a02-4437-b9fa-d27ed7792f4f`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 11:28:58,434::task::1185::TaskManager.Task::(prepare) Task=`dbf36a9d-b9a1-4745-8681-f91160224cb6`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000698492', 'lastCheck': '2.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:28:58,435::task::595::TaskManager.Task::(_updateState) Task=`dbf36a9d-b9a1-4745-8681-f91160224cb6`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:28:58,436::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:28:58,436::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:28:58,438::task::990::TaskManager.Task::(_decref) Task=`dbf36a9d-b9a1-4745-8681-f91160224cb6`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:28:58,442::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:28:58,442::task::595::TaskManager.Task::(_updateState) Task=`3636b9df-8317-46ff-b687-0ad18bad5135`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:28:58,443::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:28:58,443::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6726831d-8cf9-4721-a1a6-e4b746c90fcf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:28:58,443::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:28:58,443::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:28:58,444::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6726831d-8cf9-4721-a1a6-e4b746c90fcf`::Granted request >Thread-338::DEBUG::2014-05-13 11:28:58,444::task::827::TaskManager.Task::(resourceAcquired) Task=`3636b9df-8317-46ff-b687-0ad18bad5135`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:28:58,444::task::990::TaskManager.Task::(_decref) Task=`3636b9df-8317-46ff-b687-0ad18bad5135`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:28:58,445::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:28:58,445::task::1185::TaskManager.Task::(prepare) Task=`3636b9df-8317-46ff-b687-0ad18bad5135`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:28:58,445::task::595::TaskManager.Task::(_updateState) Task=`3636b9df-8317-46ff-b687-0ad18bad5135`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:28:58,445::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:28:58,446::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:28:58,446::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:28:58,446::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:28:58,446::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:28:58,446::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:28:58,447::task::990::TaskManager.Task::(_decref) Task=`3636b9df-8317-46ff-b687-0ad18bad5135`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:28:58,549::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:28:58,629::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365606 s, 28.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:28:58,846::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:29:00,640::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:00,757::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.060132 s, 17.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:29:01,864::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:29:02,770::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:02,849::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356189 s, 28.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:29:04,862::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:29:04,882::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:29:04,955::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0493582 s, 20.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:29:06,444::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:29:06,453::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000625122 s, 6.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:29:06,968::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:07,046::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035173 s, 29.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:29:07,904::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:29:08,472::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:29:08,473::task::595::TaskManager.Task::(_updateState) Task=`9422addf-b023-4754-ba41-e9ea467e9a34`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:29:08,473::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:29:08,473::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:29:08,474::task::1185::TaskManager.Task::(prepare) Task=`9422addf-b023-4754-ba41-e9ea467e9a34`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:29:08,474::task::595::TaskManager.Task::(_updateState) Task=`9422addf-b023-4754-ba41-e9ea467e9a34`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:29:08,474::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:29:08,474::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:29:08,474::task::990::TaskManager.Task::(_decref) Task=`9422addf-b023-4754-ba41-e9ea467e9a34`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:29:08,479::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:29:08,479::task::595::TaskManager.Task::(_updateState) Task=`6c24cc9c-fa07-456b-9fab-d8f263fbbe5c`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:29:08,480::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:29:08,481::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ba44459f-24dd-4267-9de6-a6e28bb1dacb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:29:08,481::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:29:08,481::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:29:08,481::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ba44459f-24dd-4267-9de6-a6e28bb1dacb`::Granted request >Thread-338::DEBUG::2014-05-13 11:29:08,482::task::827::TaskManager.Task::(resourceAcquired) Task=`6c24cc9c-fa07-456b-9fab-d8f263fbbe5c`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:29:08,482::task::990::TaskManager.Task::(_decref) Task=`6c24cc9c-fa07-456b-9fab-d8f263fbbe5c`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:29:08,483::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:29:08,483::task::1185::TaskManager.Task::(prepare) Task=`6c24cc9c-fa07-456b-9fab-d8f263fbbe5c`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:29:08,483::task::595::TaskManager.Task::(_updateState) Task=`6c24cc9c-fa07-456b-9fab-d8f263fbbe5c`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:29:08,483::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:29:08,483::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:29:08,483::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:29:08,484::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:29:08,484::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:29:08,484::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:29:08,484::task::990::TaskManager.Task::(_decref) Task=`6c24cc9c-fa07-456b-9fab-d8f263fbbe5c`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:29:09,059::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:09,133::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0298995 s, 34.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:29:10,925::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:29:11,146::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:11,221::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0328792 s, 31.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:29:13,234::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:13,312::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0348339 s, 29.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:29:13,581::task::595::TaskManager.Task::(_updateState) Task=`548311e2-7a19-4ebd-a4b9-c18479e12443`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:29:13,581::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:29:13,581::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000625122', 'lastCheck': '7.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:29:13,581::task::1185::TaskManager.Task::(prepare) Task=`548311e2-7a19-4ebd-a4b9-c18479e12443`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000625122', 'lastCheck': '7.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:29:13,581::task::595::TaskManager.Task::(_updateState) Task=`548311e2-7a19-4ebd-a4b9-c18479e12443`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:29:13,582::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:29:13,582::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:29:13,582::task::990::TaskManager.Task::(_decref) Task=`548311e2-7a19-4ebd-a4b9-c18479e12443`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:29:13,944::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:29:15,325::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:15,411::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0420539 s, 24.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:29:16,460::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:29:16,469::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000665301 s, 6.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:29:16,964::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:29:17,425::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:17,498::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0299088 s, 34.2 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:29:18,505::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:29:18,505::task::595::TaskManager.Task::(_updateState) Task=`43e0e9e9-6c0b-40da-ad33-7261c72c3767`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:29:18,505::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:29:18,506::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:29:18,506::task::1185::TaskManager.Task::(prepare) Task=`43e0e9e9-6c0b-40da-ad33-7261c72c3767`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:29:18,506::task::595::TaskManager.Task::(_updateState) Task=`43e0e9e9-6c0b-40da-ad33-7261c72c3767`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:29:18,506::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:29:18,507::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:29:18,507::task::990::TaskManager.Task::(_decref) Task=`43e0e9e9-6c0b-40da-ad33-7261c72c3767`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:29:18,511::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:29:18,512::task::595::TaskManager.Task::(_updateState) Task=`c6ea783c-8d9c-41fb-a665-6916f91fd503`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:29:18,512::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:29:18,512::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`84e9809c-4143-424b-bd28-2f9e426849cb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:29:18,512::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:29:18,513::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:29:18,513::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`84e9809c-4143-424b-bd28-2f9e426849cb`::Granted request >Thread-338::DEBUG::2014-05-13 11:29:18,513::task::827::TaskManager.Task::(resourceAcquired) Task=`c6ea783c-8d9c-41fb-a665-6916f91fd503`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:29:18,513::task::990::TaskManager.Task::(_decref) Task=`c6ea783c-8d9c-41fb-a665-6916f91fd503`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:29:18,514::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:29:18,514::task::1185::TaskManager.Task::(prepare) Task=`c6ea783c-8d9c-41fb-a665-6916f91fd503`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:29:18,515::task::595::TaskManager.Task::(_updateState) Task=`c6ea783c-8d9c-41fb-a665-6916f91fd503`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:29:18,515::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:29:18,515::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:29:18,515::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:29:18,515::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:29:18,515::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:29:18,516::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:29:18,516::task::990::TaskManager.Task::(_decref) Task=`c6ea783c-8d9c-41fb-a665-6916f91fd503`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:29:19,510::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:19,587::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0346978 s, 29.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:29:19,984::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:29:21,600::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:21,682::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0389903 s, 26.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:29:23,002::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:29:23,695::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:23,769::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0299716 s, 34.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:29:25,782::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:25,860::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0348248 s, 29.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:29:26,020::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:29:26,475::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:29:26,484::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000682458 s, 6.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:29:27,873::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:28,287::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.37046 s, 2.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:29:28,539::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:29:28,539::task::595::TaskManager.Task::(_updateState) Task=`e04bc8fc-57d7-493e-8525-c8732fb22580`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:29:28,539::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:29:28,540::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:29:28,540::task::1185::TaskManager.Task::(prepare) Task=`e04bc8fc-57d7-493e-8525-c8732fb22580`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:29:28,540::task::595::TaskManager.Task::(_updateState) Task=`e04bc8fc-57d7-493e-8525-c8732fb22580`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:29:28,540::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:29:28,540::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:29:28,540::task::990::TaskManager.Task::(_decref) Task=`e04bc8fc-57d7-493e-8525-c8732fb22580`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:29:28,546::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:29:28,546::task::595::TaskManager.Task::(_updateState) Task=`97f00754-8758-4c0f-94f1-5218b9cfbcd8`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:29:28,546::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:29:28,546::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5ce7992f-69cb-47af-8c31-160f127b2499`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:29:28,547::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:29:28,547::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:29:28,547::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5ce7992f-69cb-47af-8c31-160f127b2499`::Granted request >Thread-338::DEBUG::2014-05-13 11:29:28,547::task::827::TaskManager.Task::(resourceAcquired) Task=`97f00754-8758-4c0f-94f1-5218b9cfbcd8`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:29:28,548::task::990::TaskManager.Task::(_decref) Task=`97f00754-8758-4c0f-94f1-5218b9cfbcd8`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:29:28,548::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:29:28,549::task::1185::TaskManager.Task::(prepare) Task=`97f00754-8758-4c0f-94f1-5218b9cfbcd8`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:29:28,549::task::595::TaskManager.Task::(_updateState) Task=`97f00754-8758-4c0f-94f1-5218b9cfbcd8`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:29:28,549::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:29:28,549::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:29:28,549::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:29:28,550::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:29:28,550::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:29:28,550::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:29:28,550::task::990::TaskManager.Task::(_decref) Task=`97f00754-8758-4c0f-94f1-5218b9cfbcd8`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 11:29:28,710::task::595::TaskManager.Task::(_updateState) Task=`4de9f788-c9f7-4d85-9ea4-48cb3574d6e4`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:29:28,710::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:29:28,711::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000682458', 'lastCheck': '2.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:29:28,711::task::1185::TaskManager.Task::(prepare) Task=`4de9f788-c9f7-4d85-9ea4-48cb3574d6e4`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000682458', 'lastCheck': '2.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:29:28,711::task::595::TaskManager.Task::(_updateState) Task=`4de9f788-c9f7-4d85-9ea4-48cb3574d6e4`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:29:28,711::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:29:28,711::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:29:28,711::task::990::TaskManager.Task::(_decref) Task=`4de9f788-c9f7-4d85-9ea4-48cb3574d6e4`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:29:29,039::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:29:30,300::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:30,376::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310219 s, 33.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:29:32,056::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:29:32,389::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:32,462::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303213 s, 33.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:29:34,475::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:34,555::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0378836 s, 27.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:29:35,076::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:29:36,491::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:29:36,500::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000513577 s, 8.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:29:36,568::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:36,647::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0349849 s, 29.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:29:38,096::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:29:38,571::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:29:38,572::task::595::TaskManager.Task::(_updateState) Task=`c0bf6b5f-df49-4495-b765-b0c27034ba6b`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:29:38,572::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:29:38,572::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:29:38,572::task::1185::TaskManager.Task::(prepare) Task=`c0bf6b5f-df49-4495-b765-b0c27034ba6b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:29:38,573::task::595::TaskManager.Task::(_updateState) Task=`c0bf6b5f-df49-4495-b765-b0c27034ba6b`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:29:38,573::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:29:38,573::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:29:38,573::task::990::TaskManager.Task::(_decref) Task=`c0bf6b5f-df49-4495-b765-b0c27034ba6b`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:29:38,578::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:29:38,578::task::595::TaskManager.Task::(_updateState) Task=`cf210da6-f19d-4ebf-a133-edee5d0ac28b`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:29:38,578::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:29:38,579::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c88e4388-28e5-4a92-ade3-e303806904fe`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:29:38,579::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:29:38,579::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:29:38,579::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c88e4388-28e5-4a92-ade3-e303806904fe`::Granted request >Thread-338::DEBUG::2014-05-13 11:29:38,580::task::827::TaskManager.Task::(resourceAcquired) Task=`cf210da6-f19d-4ebf-a133-edee5d0ac28b`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:29:38,580::task::990::TaskManager.Task::(_decref) Task=`cf210da6-f19d-4ebf-a133-edee5d0ac28b`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:29:38,580::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:29:38,581::task::1185::TaskManager.Task::(prepare) Task=`cf210da6-f19d-4ebf-a133-edee5d0ac28b`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:29:38,581::task::595::TaskManager.Task::(_updateState) Task=`cf210da6-f19d-4ebf-a133-edee5d0ac28b`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:29:38,581::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:29:38,581::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:29:38,581::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:29:38,582::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:29:38,582::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:29:38,582::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:29:38,582::task::990::TaskManager.Task::(_decref) Task=`cf210da6-f19d-4ebf-a133-edee5d0ac28b`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:29:38,660::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:38,734::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0299317 s, 34.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:29:40,747::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:40,853::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0464945 s, 22.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:29:41,118::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:29:42,867::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:42,952::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0419855 s, 24.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:29:43,849::task::595::TaskManager.Task::(_updateState) Task=`89f11b88-7e0c-43f5-aa07-2256c026f286`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:29:43,849::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:29:43,849::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000513577', 'lastCheck': '7.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:29:43,850::task::1185::TaskManager.Task::(prepare) Task=`89f11b88-7e0c-43f5-aa07-2256c026f286`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000513577', 'lastCheck': '7.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:29:43,850::task::595::TaskManager.Task::(_updateState) Task=`89f11b88-7e0c-43f5-aa07-2256c026f286`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:29:43,850::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:29:43,850::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:29:43,850::task::990::TaskManager.Task::(_decref) Task=`89f11b88-7e0c-43f5-aa07-2256c026f286`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:29:44,137::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:29:44,965::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:45,038::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0301784 s, 33.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:29:46,507::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:29:46,515::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000643591 s, 6.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:29:47,051::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:47,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0401431 s, 25.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:29:47,155::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:29:48,607::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1c80df6] >Thread-66::DEBUG::2014-05-13 11:29:48,607::task::595::TaskManager.Task::(_updateState) Task=`183a730d-c616-43ab-851c-6c0d710d00e6`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:29:48,607::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:29:48,608::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:29:48,608::task::1185::TaskManager.Task::(prepare) Task=`183a730d-c616-43ab-851c-6c0d710d00e6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:29:48,608::task::595::TaskManager.Task::(_updateState) Task=`183a730d-c616-43ab-851c-6c0d710d00e6`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:29:48,608::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:29:48,609::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:29:48,609::task::990::TaskManager.Task::(_decref) Task=`183a730d-c616-43ab-851c-6c0d710d00e6`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:29:48,614::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1c80df6] >Thread-338::DEBUG::2014-05-13 11:29:48,614::task::595::TaskManager.Task::(_updateState) Task=`6e968c2d-882a-4323-ae87-2e9443bdb1d4`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:29:48,614::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:29:48,615::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9c41d8ed-57ac-4174-b979-c99cb028c072`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:29:48,615::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:29:48,615::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:29:48,615::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9c41d8ed-57ac-4174-b979-c99cb028c072`::Granted request >Thread-338::DEBUG::2014-05-13 11:29:48,615::task::827::TaskManager.Task::(resourceAcquired) Task=`6e968c2d-882a-4323-ae87-2e9443bdb1d4`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:29:48,616::task::990::TaskManager.Task::(_decref) Task=`6e968c2d-882a-4323-ae87-2e9443bdb1d4`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:29:48,616::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:29:48,617::task::1185::TaskManager.Task::(prepare) Task=`6e968c2d-882a-4323-ae87-2e9443bdb1d4`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:29:48,617::task::595::TaskManager.Task::(_updateState) Task=`6e968c2d-882a-4323-ae87-2e9443bdb1d4`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:29:48,617::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:29:48,617::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:29:48,617::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:29:48,618::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:29:48,618::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:29:48,618::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:29:48,618::task::990::TaskManager.Task::(_decref) Task=`6e968c2d-882a-4323-ae87-2e9443bdb1d4`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:29:49,148::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:49,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0370176 s, 27.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:29:50,174::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:29:51,242::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:51,317::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0304652 s, 33.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:29:53,194::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:29:53,330::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:53,404::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307093 s, 33.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:29:55,418::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:55,791::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.330097 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:29:56,220::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:29:56,521::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 41830dad-f012-4b4d-b636-863f66d5ca4b >Thread-322::DEBUG::2014-05-13 11:29:56,521::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-322::DEBUG::2014-05-13 11:29:56,524::blockSD::336::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:29:56,534::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.00218873 s, 936 kB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:29:56,534::misc::287::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.00218873 s, 936 kB/s'], size: 2048 >Thread-322::DEBUG::2014-05-13 11:29:56,534::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-322::WARNING::2014-05-13 11:29:56,534::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-322::DEBUG::2014-05-13 11:29:56,535::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-322::DEBUG::2014-05-13 11:29:56,535::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:29:56,535::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:29:56,536::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:29:56,536::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:29:56,536::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:29:56,537::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 11:29:56,761::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:29:56,762::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:29:56,763::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'LOCKPOLICY=', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', 'DESCRIPTION=ISCSI', 'LOGBLKSIZE=512', 'TYPE=ISCSI', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'PHYBLKSIZE=512', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', 'CLASS=Data', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'POOL_DESCRIPTION=DC_2', 'POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'MASTER_VERSION=1', 'ROLE=Master', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', '_SHA_CKSUM=28bb2d7cc60e0d779eb69fe93d050c16c769f069', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0'] >Thread-322::INFO::2014-05-13 11:29:56,764::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_imageNS already registered >Thread-322::INFO::2014-05-13 11:29:56,764::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_volumeNS already registered >Thread-322::INFO::2014-05-13 11:29:56,764::blockSD::452::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_lvmActivationNS already registered >Thread-322::DEBUG::2014-05-13 11:29:56,765::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 11:29:56,934::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:29:56,939::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:29:56,948::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000632938 s, 6.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:29:57,803::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:57,884::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361942 s, 28.3 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:29:58,638::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:29:58,639::task::595::TaskManager.Task::(_updateState) Task=`c3798c09-b0ca-4ecb-b08e-af7d4e7a5a5a`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:29:58,639::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:29:58,639::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:29:58,639::task::1185::TaskManager.Task::(prepare) Task=`c3798c09-b0ca-4ecb-b08e-af7d4e7a5a5a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:29:58,640::task::595::TaskManager.Task::(_updateState) Task=`c3798c09-b0ca-4ecb-b08e-af7d4e7a5a5a`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:29:58,640::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:29:58,640::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:29:58,640::task::990::TaskManager.Task::(_decref) Task=`c3798c09-b0ca-4ecb-b08e-af7d4e7a5a5a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:29:58,645::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:29:58,645::task::595::TaskManager.Task::(_updateState) Task=`7a408f0a-aead-405f-845d-0d3f432879bb`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:29:58,645::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:29:58,646::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0c267fd8-4d76-49d2-b72a-2d795871ee73`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:29:58,646::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:29:58,646::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:29:58,646::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0c267fd8-4d76-49d2-b72a-2d795871ee73`::Granted request >Thread-338::DEBUG::2014-05-13 11:29:58,646::task::827::TaskManager.Task::(resourceAcquired) Task=`7a408f0a-aead-405f-845d-0d3f432879bb`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:29:58,647::task::990::TaskManager.Task::(_decref) Task=`7a408f0a-aead-405f-845d-0d3f432879bb`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:29:58,647::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:29:58,648::task::1185::TaskManager.Task::(prepare) Task=`7a408f0a-aead-405f-845d-0d3f432879bb`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:29:58,648::task::595::TaskManager.Task::(_updateState) Task=`7a408f0a-aead-405f-845d-0d3f432879bb`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:29:58,648::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:29:58,648::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:29:58,648::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:29:58,649::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:29:58,649::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:29:58,649::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:29:58,649::task::990::TaskManager.Task::(_decref) Task=`7a408f0a-aead-405f-845d-0d3f432879bb`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 11:29:58,984::task::595::TaskManager.Task::(_updateState) Task=`aaea4726-3d4c-4659-9c9a-3b25e1401595`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:29:58,984::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:29:58,984::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000632938', 'lastCheck': '2.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:29:58,984::task::1185::TaskManager.Task::(prepare) Task=`aaea4726-3d4c-4659-9c9a-3b25e1401595`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000632938', 'lastCheck': '2.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:29:58,985::task::595::TaskManager.Task::(_updateState) Task=`aaea4726-3d4c-4659-9c9a-3b25e1401595`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:29:58,985::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:29:58,985::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:29:58,985::task::990::TaskManager.Task::(_decref) Task=`aaea4726-3d4c-4659-9c9a-3b25e1401595`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:29:59,238::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:29:59,897::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:29:59,971::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.029907 s, 34.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:30:01,984::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:30:02,063::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0348968 s, 29.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:30:02,256::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:30:04,080::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:30:04,160::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358147 s, 28.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:30:05,275::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:30:06,174::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:30:06,248::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323719 s, 31.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:30:06,955::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:30:06,964::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000632575 s, 6.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:30:08,261::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:30:08,298::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:30:08,348::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0442988 s, 23.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:30:08,669::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:30:08,669::task::595::TaskManager.Task::(_updateState) Task=`cc2284cb-ed2b-46cb-93cf-8cbfd47a1e72`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:30:08,669::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:30:08,670::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:30:08,670::task::1185::TaskManager.Task::(prepare) Task=`cc2284cb-ed2b-46cb-93cf-8cbfd47a1e72`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:30:08,670::task::595::TaskManager.Task::(_updateState) Task=`cc2284cb-ed2b-46cb-93cf-8cbfd47a1e72`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:30:08,670::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:30:08,670::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:30:08,671::task::990::TaskManager.Task::(_decref) Task=`cc2284cb-ed2b-46cb-93cf-8cbfd47a1e72`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:30:08,675::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:30:08,675::task::595::TaskManager.Task::(_updateState) Task=`b9018f3e-934b-4edb-aa7e-2b5467b8d51d`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:30:08,676::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:30:08,676::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a12f112d-9269-4a4f-8f94-69bdd04caa90`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:30:08,676::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:30:08,677::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:30:08,677::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a12f112d-9269-4a4f-8f94-69bdd04caa90`::Granted request >Thread-338::DEBUG::2014-05-13 11:30:08,677::task::827::TaskManager.Task::(resourceAcquired) Task=`b9018f3e-934b-4edb-aa7e-2b5467b8d51d`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:30:08,677::task::990::TaskManager.Task::(_decref) Task=`b9018f3e-934b-4edb-aa7e-2b5467b8d51d`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:30:08,678::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:30:08,678::task::1185::TaskManager.Task::(prepare) Task=`b9018f3e-934b-4edb-aa7e-2b5467b8d51d`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:30:08,678::task::595::TaskManager.Task::(_updateState) Task=`b9018f3e-934b-4edb-aa7e-2b5467b8d51d`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:30:08,679::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:30:08,679::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:30:08,679::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:30:08,679::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:30:08,679::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:30:08,679::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:30:08,680::task::990::TaskManager.Task::(_decref) Task=`b9018f3e-934b-4edb-aa7e-2b5467b8d51d`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:30:10,361::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:30:10,440::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0368163 s, 27.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:30:11,319::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:30:12,453::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:30:12,528::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322976 s, 31.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:30:14,122::task::595::TaskManager.Task::(_updateState) Task=`af284a22-57fe-4974-9728-e0493dc44682`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:30:14,122::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:30:14,122::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000632575', 'lastCheck': '7.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:30:14,122::task::1185::TaskManager.Task::(prepare) Task=`af284a22-57fe-4974-9728-e0493dc44682`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000632575', 'lastCheck': '7.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:30:14,122::task::595::TaskManager.Task::(_updateState) Task=`af284a22-57fe-4974-9728-e0493dc44682`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:30:14,123::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:30:14,123::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:30:14,123::task::990::TaskManager.Task::(_decref) Task=`af284a22-57fe-4974-9728-e0493dc44682`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:30:14,337::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:30:14,541::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:30:14,615::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321447 s, 31.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:30:16,628::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:30:16,706::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036788 s, 27.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:30:16,971::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:30:16,980::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000631387 s, 6.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:30:17,355::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:30:18,703::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [206acf2a] >Thread-66::DEBUG::2014-05-13 11:30:18,703::task::595::TaskManager.Task::(_updateState) Task=`663faecf-f399-475a-8b86-f26221e67879`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:30:18,703::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:30:18,704::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:30:18,704::task::1185::TaskManager.Task::(prepare) Task=`663faecf-f399-475a-8b86-f26221e67879`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:30:18,704::task::595::TaskManager.Task::(_updateState) Task=`663faecf-f399-475a-8b86-f26221e67879`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:30:18,704::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:30:18,705::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:30:18,705::task::990::TaskManager.Task::(_decref) Task=`663faecf-f399-475a-8b86-f26221e67879`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:30:18,709::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [206acf2a] >Thread-338::DEBUG::2014-05-13 11:30:18,710::task::595::TaskManager.Task::(_updateState) Task=`3339d41d-cefb-46bd-a1e3-6f748589c73e`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:30:18,710::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:30:18,710::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`678b5580-4718-4e3b-a2b1-84b907e3333e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:30:18,710::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:30:18,711::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:30:18,711::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`678b5580-4718-4e3b-a2b1-84b907e3333e`::Granted request >Thread-338::DEBUG::2014-05-13 11:30:18,711::task::827::TaskManager.Task::(resourceAcquired) Task=`3339d41d-cefb-46bd-a1e3-6f748589c73e`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:30:18,711::task::990::TaskManager.Task::(_decref) Task=`3339d41d-cefb-46bd-a1e3-6f748589c73e`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:30:18,712::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:30:18,712::task::1185::TaskManager.Task::(prepare) Task=`3339d41d-cefb-46bd-a1e3-6f748589c73e`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:30:18,713::task::595::TaskManager.Task::(_updateState) Task=`3339d41d-cefb-46bd-a1e3-6f748589c73e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:30:18,713::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:30:18,713::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:30:18,713::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:30:18,714::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:30:18,714::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:30:18,714::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:30:18,714::task::990::TaskManager.Task::(_decref) Task=`3339d41d-cefb-46bd-a1e3-6f748589c73e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:30:18,719::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:30:18,797::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0348846 s, 29.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:30:20,374::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:30:20,810::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:30:20,883::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0299101 s, 34.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:30:22,896::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:30:23,013::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0742109 s, 13.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:30:23,394::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:30:25,026::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:30:25,101::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307093 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:30:26,413::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:30:26,987::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:30:26,995::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000671401 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:30:27,114::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:30:27,193::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364637 s, 28.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:30:28,735::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:30:28,735::task::595::TaskManager.Task::(_updateState) Task=`f28d1fe0-b2bc-4d7a-b082-feb7973539f4`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:30:28,736::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:30:28,736::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:30:28,736::task::1185::TaskManager.Task::(prepare) Task=`f28d1fe0-b2bc-4d7a-b082-feb7973539f4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:30:28,736::task::595::TaskManager.Task::(_updateState) Task=`f28d1fe0-b2bc-4d7a-b082-feb7973539f4`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:30:28,737::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:30:28,737::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:30:28,737::task::990::TaskManager.Task::(_decref) Task=`f28d1fe0-b2bc-4d7a-b082-feb7973539f4`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:30:28,741::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:30:28,742::task::595::TaskManager.Task::(_updateState) Task=`cc6f01a9-095a-4fa0-b69a-becba312b532`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:30:28,742::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:30:28,742::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`85f88cbc-9052-48e3-8751-8c6cc1b0f79d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:30:28,743::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:30:28,743::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:30:28,743::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`85f88cbc-9052-48e3-8751-8c6cc1b0f79d`::Granted request >Thread-338::DEBUG::2014-05-13 11:30:28,743::task::827::TaskManager.Task::(resourceAcquired) Task=`cc6f01a9-095a-4fa0-b69a-becba312b532`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:30:28,744::task::990::TaskManager.Task::(_decref) Task=`cc6f01a9-095a-4fa0-b69a-becba312b532`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:30:28,744::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:30:28,745::task::1185::TaskManager.Task::(prepare) Task=`cc6f01a9-095a-4fa0-b69a-becba312b532`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:30:28,745::task::595::TaskManager.Task::(_updateState) Task=`cc6f01a9-095a-4fa0-b69a-becba312b532`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:30:28,745::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:30:28,745::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:30:28,745::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:30:28,745::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:30:28,746::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:30:28,746::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:30:28,746::task::990::TaskManager.Task::(_decref) Task=`cc6f01a9-095a-4fa0-b69a-becba312b532`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:30:29,207::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 11:30:29,262::task::595::TaskManager.Task::(_updateState) Task=`b4b27c03-798a-4b2e-a087-5af1a708642c`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:30:29,263::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:30:29,264::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000671401', 'lastCheck': '2.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:30:29,265::task::1185::TaskManager.Task::(prepare) Task=`b4b27c03-798a-4b2e-a087-5af1a708642c`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000671401', 'lastCheck': '2.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:30:29,266::task::595::TaskManager.Task::(_updateState) Task=`b4b27c03-798a-4b2e-a087-5af1a708642c`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:30:29,266::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:30:29,267::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:30:29,269::task::990::TaskManager.Task::(_decref) Task=`b4b27c03-798a-4b2e-a087-5af1a708642c`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:30:29,308::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.029801 s, 34.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:30:29,434::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:30:31,321::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:30:31,400::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0368854 s, 27.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:30:32,453::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:30:33,413::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:30:33,493::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356946 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:30:35,473::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:30:35,506::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:30:35,579::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0297343 s, 34.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:30:37,003::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:30:37,011::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000515045 s, 8.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:30:37,592::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:30:37,953::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.31721 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:30:38,493::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:30:38,767::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2555e502] >Thread-66::DEBUG::2014-05-13 11:30:38,767::task::595::TaskManager.Task::(_updateState) Task=`e4e616b2-6a68-446b-8de6-7cc56a4bf0a2`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:30:38,768::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:30:38,768::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:30:38,768::task::1185::TaskManager.Task::(prepare) Task=`e4e616b2-6a68-446b-8de6-7cc56a4bf0a2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:30:38,768::task::595::TaskManager.Task::(_updateState) Task=`e4e616b2-6a68-446b-8de6-7cc56a4bf0a2`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:30:38,768::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:30:38,769::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:30:38,769::task::990::TaskManager.Task::(_decref) Task=`e4e616b2-6a68-446b-8de6-7cc56a4bf0a2`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:30:38,773::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2555e502] >Thread-338::DEBUG::2014-05-13 11:30:38,774::task::595::TaskManager.Task::(_updateState) Task=`bbc46281-4051-4164-bd0b-8aff8555a18a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:30:38,774::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:30:38,774::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e5ecf7a4-4dfa-4c54-a7f8-9f0bee5ecd28`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:30:38,774::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:30:38,775::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:30:38,775::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e5ecf7a4-4dfa-4c54-a7f8-9f0bee5ecd28`::Granted request >Thread-338::DEBUG::2014-05-13 11:30:38,775::task::827::TaskManager.Task::(resourceAcquired) Task=`bbc46281-4051-4164-bd0b-8aff8555a18a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:30:38,775::task::990::TaskManager.Task::(_decref) Task=`bbc46281-4051-4164-bd0b-8aff8555a18a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:30:38,776::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:30:38,776::task::1185::TaskManager.Task::(prepare) Task=`bbc46281-4051-4164-bd0b-8aff8555a18a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:30:38,776::task::595::TaskManager.Task::(_updateState) Task=`bbc46281-4051-4164-bd0b-8aff8555a18a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:30:38,776::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:30:38,777::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:30:38,777::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:30:38,777::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:30:38,777::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:30:38,777::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:30:38,777::task::990::TaskManager.Task::(_decref) Task=`bbc46281-4051-4164-bd0b-8aff8555a18a`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:30:39,966::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:30:40,041::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310702 s, 33.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:30:41,514::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:30:42,054::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:30:42,128::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0302727 s, 33.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:30:44,141::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:30:44,221::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0368678 s, 27.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:30:44,416::task::595::TaskManager.Task::(_updateState) Task=`0e4d83de-c2ef-4a4e-8784-e9e8cff07112`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:30:44,416::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:30:44,416::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000515045', 'lastCheck': '7.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:30:44,417::task::1185::TaskManager.Task::(prepare) Task=`0e4d83de-c2ef-4a4e-8784-e9e8cff07112`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000515045', 'lastCheck': '7.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:30:44,417::task::595::TaskManager.Task::(_updateState) Task=`0e4d83de-c2ef-4a4e-8784-e9e8cff07112`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:30:44,417::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:30:44,417::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:30:44,417::task::990::TaskManager.Task::(_decref) Task=`0e4d83de-c2ef-4a4e-8784-e9e8cff07112`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:30:44,537::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:30:46,233::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:30:46,309::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311393 s, 32.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:30:47,030::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:30:47,039::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000781609 s, 5.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:30:47,564::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:30:48,322::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:30:48,403::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352356 s, 29.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:30:48,798::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:30:48,799::task::595::TaskManager.Task::(_updateState) Task=`bdadf933-847f-41b2-97d8-fa7b61133452`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:30:48,799::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:30:48,799::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:30:48,799::task::1185::TaskManager.Task::(prepare) Task=`bdadf933-847f-41b2-97d8-fa7b61133452`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:30:48,800::task::595::TaskManager.Task::(_updateState) Task=`bdadf933-847f-41b2-97d8-fa7b61133452`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:30:48,800::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:30:48,800::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:30:48,800::task::990::TaskManager.Task::(_decref) Task=`bdadf933-847f-41b2-97d8-fa7b61133452`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:30:48,806::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:30:48,806::task::595::TaskManager.Task::(_updateState) Task=`af0eac51-1adb-492e-b62d-8110fadc54f2`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:30:48,806::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:30:48,807::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d294b1b9-c458-417f-a911-50c12320dbf5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:30:48,807::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:30:48,807::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:30:48,808::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d294b1b9-c458-417f-a911-50c12320dbf5`::Granted request >Thread-338::DEBUG::2014-05-13 11:30:48,808::task::827::TaskManager.Task::(resourceAcquired) Task=`af0eac51-1adb-492e-b62d-8110fadc54f2`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:30:48,808::task::990::TaskManager.Task::(_decref) Task=`af0eac51-1adb-492e-b62d-8110fadc54f2`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:30:48,809::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:30:48,809::task::1185::TaskManager.Task::(prepare) Task=`af0eac51-1adb-492e-b62d-8110fadc54f2`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:30:48,809::task::595::TaskManager.Task::(_updateState) Task=`af0eac51-1adb-492e-b62d-8110fadc54f2`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:30:48,809::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:30:48,810::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:30:48,810::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:30:48,810::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:30:48,810::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:30:48,810::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:30:48,811::task::990::TaskManager.Task::(_decref) Task=`af0eac51-1adb-492e-b62d-8110fadc54f2`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:30:50,416::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:30:50,588::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:30:50,780::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.322297 s, 3.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:30:52,793::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:30:52,872::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352744 s, 29.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:30:53,606::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:30:54,885::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:30:54,959::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0295774 s, 34.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:30:56,629::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:30:56,971::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:30:57,073::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305844 s, 33.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:30:57,074::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:30:57,097::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000684793 s, 6.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:30:58,831::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [30b3ccf] >Thread-66::DEBUG::2014-05-13 11:30:58,832::task::595::TaskManager.Task::(_updateState) Task=`0d363bc2-739c-4f54-87f7-0781d5137bf8`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:30:58,832::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:30:58,833::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:30:58,833::task::1185::TaskManager.Task::(prepare) Task=`0d363bc2-739c-4f54-87f7-0781d5137bf8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:30:58,833::task::595::TaskManager.Task::(_updateState) Task=`0d363bc2-739c-4f54-87f7-0781d5137bf8`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:30:58,833::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:30:58,833::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:30:58,833::task::990::TaskManager.Task::(_decref) Task=`0d363bc2-739c-4f54-87f7-0781d5137bf8`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:30:58,838::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [30b3ccf] >Thread-338::DEBUG::2014-05-13 11:30:58,839::task::595::TaskManager.Task::(_updateState) Task=`aa61374d-986a-45af-87ff-14bc2d81c541`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:30:58,839::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:30:58,839::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`405243c5-dc6b-44fd-a721-9eaaa1003459`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:30:58,839::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:30:58,840::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:30:58,840::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`405243c5-dc6b-44fd-a721-9eaaa1003459`::Granted request >Thread-338::DEBUG::2014-05-13 11:30:58,840::task::827::TaskManager.Task::(resourceAcquired) Task=`aa61374d-986a-45af-87ff-14bc2d81c541`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:30:58,840::task::990::TaskManager.Task::(_decref) Task=`aa61374d-986a-45af-87ff-14bc2d81c541`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:30:58,841::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:30:58,841::task::1185::TaskManager.Task::(prepare) Task=`aa61374d-986a-45af-87ff-14bc2d81c541`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:30:58,842::task::595::TaskManager.Task::(_updateState) Task=`aa61374d-986a-45af-87ff-14bc2d81c541`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:30:58,842::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:30:58,842::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:30:58,842::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:30:58,842::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:30:58,843::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:30:58,843::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:30:58,843::task::990::TaskManager.Task::(_decref) Task=`aa61374d-986a-45af-87ff-14bc2d81c541`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:30:59,130::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:30:59,210::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356911 s, 28.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:30:59,551::task::595::TaskManager.Task::(_updateState) Task=`4ff798d2-1a3b-4f0e-922e-e00fd3aba221`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:30:59,551::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:30:59,552::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000684793', 'lastCheck': '2.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:30:59,552::task::1185::TaskManager.Task::(prepare) Task=`4ff798d2-1a3b-4f0e-922e-e00fd3aba221`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000684793', 'lastCheck': '2.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:30:59,552::task::595::TaskManager.Task::(_updateState) Task=`4ff798d2-1a3b-4f0e-922e-e00fd3aba221`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:30:59,552::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:30:59,552::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:30:59,552::task::990::TaskManager.Task::(_decref) Task=`4ff798d2-1a3b-4f0e-922e-e00fd3aba221`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:30:59,648::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:31:01,223::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:31:01,308::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.041948 s, 24.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:31:02,666::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:31:03,321::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:31:03,416::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0503585 s, 20.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:31:05,430::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:31:05,501::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0295813 s, 34.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:31:05,683::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:31:07,121::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:31:07,130::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000593359 s, 6.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:31:07,514::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:31:07,593::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361525 s, 28.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:31:08,703::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:31:08,861::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [72471f8f] >Thread-66::DEBUG::2014-05-13 11:31:08,861::task::595::TaskManager.Task::(_updateState) Task=`ff3c3cbe-24a2-44ca-9055-c56afd5f3754`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:31:08,862::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:31:08,862::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:31:08,862::task::1185::TaskManager.Task::(prepare) Task=`ff3c3cbe-24a2-44ca-9055-c56afd5f3754`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:31:08,862::task::595::TaskManager.Task::(_updateState) Task=`ff3c3cbe-24a2-44ca-9055-c56afd5f3754`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:31:08,862::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:31:08,862::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:31:08,863::task::990::TaskManager.Task::(_decref) Task=`ff3c3cbe-24a2-44ca-9055-c56afd5f3754`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:31:08,868::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [72471f8f] >Thread-338::DEBUG::2014-05-13 11:31:08,868::task::595::TaskManager.Task::(_updateState) Task=`e79f25eb-3a92-464b-9dfa-c39c243d0ba4`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:31:08,868::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:31:08,869::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`09d9255d-f233-47cf-a83f-2d168fd9e656`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:31:08,869::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:31:08,869::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:31:08,869::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`09d9255d-f233-47cf-a83f-2d168fd9e656`::Granted request >Thread-338::DEBUG::2014-05-13 11:31:08,870::task::827::TaskManager.Task::(resourceAcquired) Task=`e79f25eb-3a92-464b-9dfa-c39c243d0ba4`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:31:08,870::task::990::TaskManager.Task::(_decref) Task=`e79f25eb-3a92-464b-9dfa-c39c243d0ba4`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:31:08,870::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:31:08,871::task::1185::TaskManager.Task::(prepare) Task=`e79f25eb-3a92-464b-9dfa-c39c243d0ba4`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:31:08,871::task::595::TaskManager.Task::(_updateState) Task=`e79f25eb-3a92-464b-9dfa-c39c243d0ba4`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:31:08,871::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:31:08,871::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:31:08,871::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:31:08,872::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:31:08,872::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:31:08,872::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:31:08,872::task::990::TaskManager.Task::(_decref) Task=`e79f25eb-3a92-464b-9dfa-c39c243d0ba4`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:31:09,606::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:31:09,680::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303733 s, 33.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:31:11,693::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:31:11,723::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:31:11,768::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0325649 s, 31.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:31:13,781::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:31:13,860::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355584 s, 28.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:31:14,693::task::595::TaskManager.Task::(_updateState) Task=`dc41681d-325e-4b7a-afca-97232a6f9d4b`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:31:14,693::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:31:14,693::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000593359', 'lastCheck': '7.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:31:14,693::task::1185::TaskManager.Task::(prepare) Task=`dc41681d-325e-4b7a-afca-97232a6f9d4b`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000593359', 'lastCheck': '7.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:31:14,694::task::595::TaskManager.Task::(_updateState) Task=`dc41681d-325e-4b7a-afca-97232a6f9d4b`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:31:14,694::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:31:14,694::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:31:14,694::task::990::TaskManager.Task::(_decref) Task=`dc41681d-325e-4b7a-afca-97232a6f9d4b`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:31:14,742::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:31:15,873::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:31:15,947::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306518 s, 33.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:31:17,137::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:31:17,153::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00065423 s, 6.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:31:17,765::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:31:17,960::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:31:18,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0451449 s, 22.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:31:18,892::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3256e43f] >Thread-66::DEBUG::2014-05-13 11:31:18,893::task::595::TaskManager.Task::(_updateState) Task=`14ffcea8-d383-41c2-9493-220a9fe6467a`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:31:18,893::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:31:18,893::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:31:18,893::task::1185::TaskManager.Task::(prepare) Task=`14ffcea8-d383-41c2-9493-220a9fe6467a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:31:18,894::task::595::TaskManager.Task::(_updateState) Task=`14ffcea8-d383-41c2-9493-220a9fe6467a`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:31:18,894::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:31:18,894::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:31:18,894::task::990::TaskManager.Task::(_decref) Task=`14ffcea8-d383-41c2-9493-220a9fe6467a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:31:18,898::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3256e43f] >Thread-338::DEBUG::2014-05-13 11:31:18,899::task::595::TaskManager.Task::(_updateState) Task=`44e9cf5e-d3c6-48a3-a020-44b7c65ae6d4`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:31:18,899::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:31:18,899::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`07e663b3-c3bf-435f-9fc3-595eece27cfa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:31:18,900::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:31:18,900::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:31:18,900::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`07e663b3-c3bf-435f-9fc3-595eece27cfa`::Granted request >Thread-338::DEBUG::2014-05-13 11:31:18,900::task::827::TaskManager.Task::(resourceAcquired) Task=`44e9cf5e-d3c6-48a3-a020-44b7c65ae6d4`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:31:18,901::task::990::TaskManager.Task::(_decref) Task=`44e9cf5e-d3c6-48a3-a020-44b7c65ae6d4`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:31:18,901::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:31:18,902::task::1185::TaskManager.Task::(prepare) Task=`44e9cf5e-d3c6-48a3-a020-44b7c65ae6d4`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:31:18,902::task::595::TaskManager.Task::(_updateState) Task=`44e9cf5e-d3c6-48a3-a020-44b7c65ae6d4`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:31:18,902::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:31:18,902::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:31:18,902::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:31:18,903::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:31:18,903::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:31:18,903::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:31:18,903::task::990::TaskManager.Task::(_decref) Task=`44e9cf5e-d3c6-48a3-a020-44b7c65ae6d4`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:31:20,061::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:31:20,139::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0349692 s, 29.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:31:20,784::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:31:22,152::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:31:22,226::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0297568 s, 34.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:31:23,803::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:31:24,239::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:31:24,315::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312262 s, 32.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:31:26,327::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:31:26,402::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305461 s, 33.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:31:26,823::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:31:27,179::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:31:27,194::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000560478 s, 7.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:31:28,415::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:31:28,493::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0348267 s, 29.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:31:28,921::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:31:28,932::task::595::TaskManager.Task::(_updateState) Task=`00ebb7a7-4272-4e39-8605-d015283da7dd`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:31:28,932::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:31:28,932::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:31:28,932::task::1185::TaskManager.Task::(prepare) Task=`00ebb7a7-4272-4e39-8605-d015283da7dd`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:31:28,933::task::595::TaskManager.Task::(_updateState) Task=`00ebb7a7-4272-4e39-8605-d015283da7dd`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:31:28,933::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:31:28,933::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:31:28,933::task::990::TaskManager.Task::(_decref) Task=`00ebb7a7-4272-4e39-8605-d015283da7dd`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:31:28,938::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:31:28,938::task::595::TaskManager.Task::(_updateState) Task=`c825d744-d6b8-4885-a647-55e1dc12d9e7`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:31:28,938::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:31:28,940::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`414a903b-e06d-468a-b2e5-b54999a09d60`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:31:28,940::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:31:28,940::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:31:28,941::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`414a903b-e06d-468a-b2e5-b54999a09d60`::Granted request >Thread-338::DEBUG::2014-05-13 11:31:28,941::task::827::TaskManager.Task::(resourceAcquired) Task=`c825d744-d6b8-4885-a647-55e1dc12d9e7`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:31:28,941::task::990::TaskManager.Task::(_decref) Task=`c825d744-d6b8-4885-a647-55e1dc12d9e7`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:31:28,942::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:31:28,942::task::1185::TaskManager.Task::(prepare) Task=`c825d744-d6b8-4885-a647-55e1dc12d9e7`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:31:28,942::task::595::TaskManager.Task::(_updateState) Task=`c825d744-d6b8-4885-a647-55e1dc12d9e7`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:31:28,942::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:31:28,943::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:31:28,943::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:31:28,943::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:31:28,943::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:31:28,943::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:31:28,944::task::990::TaskManager.Task::(_decref) Task=`c825d744-d6b8-4885-a647-55e1dc12d9e7`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 11:31:29,829::task::595::TaskManager.Task::(_updateState) Task=`1ab9a559-1f9c-43fe-8001-a2c2b0696d96`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:31:29,829::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:31:29,829::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000560478', 'lastCheck': '2.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:31:29,829::task::1185::TaskManager.Task::(prepare) Task=`1ab9a559-1f9c-43fe-8001-a2c2b0696d96`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000560478', 'lastCheck': '2.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:31:29,829::task::595::TaskManager.Task::(_updateState) Task=`1ab9a559-1f9c-43fe-8001-a2c2b0696d96`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:31:29,829::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:31:29,830::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:31:29,830::task::990::TaskManager.Task::(_decref) Task=`1ab9a559-1f9c-43fe-8001-a2c2b0696d96`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:31:29,841::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:31:30,506::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:31:30,591::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0422006 s, 24.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:31:32,604::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:31:32,678::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314237 s, 32.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:31:32,859::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:31:34,691::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:31:34,771::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0369088 s, 27.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:31:35,878::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:31:36,784::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:31:36,861::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.033002 s, 31.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:31:37,207::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:31:37,216::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000663685 s, 6.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:31:38,874::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:31:38,896::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:31:38,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0331601 s, 30.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:31:38,963::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:31:38,965::task::595::TaskManager.Task::(_updateState) Task=`e5658955-acaf-43aa-8ab1-d3c7b3415e68`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:31:38,966::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:31:38,968::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:31:38,969::task::1185::TaskManager.Task::(prepare) Task=`e5658955-acaf-43aa-8ab1-d3c7b3415e68`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:31:38,969::task::595::TaskManager.Task::(_updateState) Task=`e5658955-acaf-43aa-8ab1-d3c7b3415e68`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:31:38,970::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:31:38,971::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:31:38,971::task::990::TaskManager.Task::(_decref) Task=`e5658955-acaf-43aa-8ab1-d3c7b3415e68`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:31:38,978::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:31:38,978::task::595::TaskManager.Task::(_updateState) Task=`c92b2c51-ab06-4d71-aa57-db46e1492329`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:31:38,978::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:31:38,979::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`cea35f4b-e701-4f2f-8e3e-415f40cd405a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:31:38,979::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:31:38,979::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:31:38,979::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`cea35f4b-e701-4f2f-8e3e-415f40cd405a`::Granted request >Thread-338::DEBUG::2014-05-13 11:31:38,980::task::827::TaskManager.Task::(resourceAcquired) Task=`c92b2c51-ab06-4d71-aa57-db46e1492329`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:31:38,980::task::990::TaskManager.Task::(_decref) Task=`c92b2c51-ab06-4d71-aa57-db46e1492329`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:31:38,981::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:31:38,981::task::1185::TaskManager.Task::(prepare) Task=`c92b2c51-ab06-4d71-aa57-db46e1492329`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:31:38,981::task::595::TaskManager.Task::(_updateState) Task=`c92b2c51-ab06-4d71-aa57-db46e1492329`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:31:38,981::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:31:38,981::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:31:38,982::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:31:38,982::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:31:38,982::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:31:38,982::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:31:38,982::task::990::TaskManager.Task::(_decref) Task=`c92b2c51-ab06-4d71-aa57-db46e1492329`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:31:40,978::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:31:41,057::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356429 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:31:41,914::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:31:43,070::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:31:43,145::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0308068 s, 33.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:31:44,934::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:31:44,991::task::595::TaskManager.Task::(_updateState) Task=`1fbd14b4-d788-4d78-acc8-d5cc8ce7f775`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:31:44,991::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:31:44,991::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000663685', 'lastCheck': '7.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:31:44,991::task::1185::TaskManager.Task::(prepare) Task=`1fbd14b4-d788-4d78-acc8-d5cc8ce7f775`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000663685', 'lastCheck': '7.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:31:44,992::task::595::TaskManager.Task::(_updateState) Task=`1fbd14b4-d788-4d78-acc8-d5cc8ce7f775`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:31:44,992::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:31:44,992::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:31:44,992::task::990::TaskManager.Task::(_decref) Task=`1fbd14b4-d788-4d78-acc8-d5cc8ce7f775`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:31:45,158::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:31:45,516::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.31473 s, 3.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:31:47,223::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:31:47,231::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000675112 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:31:47,529::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:31:47,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0299728 s, 34.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:31:47,964::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:31:49,003::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [dc1f215] >Thread-13::DEBUG::2014-05-13 11:31:49,003::task::595::TaskManager.Task::(_updateState) Task=`30408e6a-ab4a-432c-a92e-074dce33a865`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:31:49,003::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:31:49,004::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:31:49,004::task::1185::TaskManager.Task::(prepare) Task=`30408e6a-ab4a-432c-a92e-074dce33a865`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:31:49,004::task::595::TaskManager.Task::(_updateState) Task=`30408e6a-ab4a-432c-a92e-074dce33a865`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:31:49,004::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:31:49,004::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:31:49,005::task::990::TaskManager.Task::(_decref) Task=`30408e6a-ab4a-432c-a92e-074dce33a865`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:31:49,009::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [dc1f215] >Thread-338::DEBUG::2014-05-13 11:31:49,009::task::595::TaskManager.Task::(_updateState) Task=`12ec9e98-eb84-47a6-b786-354ebd347dd2`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:31:49,010::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:31:49,010::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1de67b02-7e6d-48e0-bd9f-c28874bb7bf1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:31:49,010::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:31:49,011::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:31:49,011::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1de67b02-7e6d-48e0-bd9f-c28874bb7bf1`::Granted request >Thread-338::DEBUG::2014-05-13 11:31:49,011::task::827::TaskManager.Task::(resourceAcquired) Task=`12ec9e98-eb84-47a6-b786-354ebd347dd2`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:31:49,011::task::990::TaskManager.Task::(_decref) Task=`12ec9e98-eb84-47a6-b786-354ebd347dd2`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:31:49,012::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:31:49,012::task::1185::TaskManager.Task::(prepare) Task=`12ec9e98-eb84-47a6-b786-354ebd347dd2`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:31:49,012::task::595::TaskManager.Task::(_updateState) Task=`12ec9e98-eb84-47a6-b786-354ebd347dd2`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:31:49,013::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:31:49,013::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:31:49,013::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:31:49,013::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:31:49,013::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:31:49,013::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:31:49,014::task::990::TaskManager.Task::(_decref) Task=`12ec9e98-eb84-47a6-b786-354ebd347dd2`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:31:49,615::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:31:49,703::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0449861 s, 22.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:31:50,985::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:31:51,716::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:31:51,791::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314884 s, 32.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:31:53,804::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:31:53,884::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035737 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:31:54,004::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:31:55,897::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:31:55,975::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0351275 s, 29.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:31:57,022::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:31:57,238::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:31:57,247::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00118655 s, 3.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:31:57,988::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:31:58,436::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.4051 s, 2.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:31:59,035::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:31:59,035::task::595::TaskManager.Task::(_updateState) Task=`47a819ae-4abd-463b-b375-6c74ae6dc127`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:31:59,035::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:31:59,036::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:31:59,036::task::1185::TaskManager.Task::(prepare) Task=`47a819ae-4abd-463b-b375-6c74ae6dc127`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:31:59,036::task::595::TaskManager.Task::(_updateState) Task=`47a819ae-4abd-463b-b375-6c74ae6dc127`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:31:59,036::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:31:59,036::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:31:59,037::task::990::TaskManager.Task::(_decref) Task=`47a819ae-4abd-463b-b375-6c74ae6dc127`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:31:59,041::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:31:59,041::task::595::TaskManager.Task::(_updateState) Task=`ea7bc575-5926-44f7-b768-31a300003553`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:31:59,042::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:31:59,042::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`7a7663d5-19f0-4e84-a768-ba54b07c32da`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:31:59,042::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:31:59,043::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:31:59,043::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`7a7663d5-19f0-4e84-a768-ba54b07c32da`::Granted request >Thread-338::DEBUG::2014-05-13 11:31:59,043::task::827::TaskManager.Task::(resourceAcquired) Task=`ea7bc575-5926-44f7-b768-31a300003553`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:31:59,043::task::990::TaskManager.Task::(_decref) Task=`ea7bc575-5926-44f7-b768-31a300003553`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:31:59,044::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:31:59,044::task::1185::TaskManager.Task::(prepare) Task=`ea7bc575-5926-44f7-b768-31a300003553`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:31:59,044::task::595::TaskManager.Task::(_updateState) Task=`ea7bc575-5926-44f7-b768-31a300003553`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:31:59,045::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:31:59,045::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:31:59,045::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:31:59,045::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:31:59,045::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:31:59,045::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:31:59,046::task::990::TaskManager.Task::(_decref) Task=`ea7bc575-5926-44f7-b768-31a300003553`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:32:00,040::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:32:00,136::task::595::TaskManager.Task::(_updateState) Task=`d6f36119-a1d8-4e99-9470-2605d1a83027`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:32:00,136::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:32:00,136::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00118655', 'lastCheck': '2.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:32:00,136::task::1185::TaskManager.Task::(prepare) Task=`d6f36119-a1d8-4e99-9470-2605d1a83027`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00118655', 'lastCheck': '2.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:32:00,136::task::595::TaskManager.Task::(_updateState) Task=`d6f36119-a1d8-4e99-9470-2605d1a83027`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:32:00,136::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:32:00,137::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:32:00,137::task::990::TaskManager.Task::(_decref) Task=`d6f36119-a1d8-4e99-9470-2605d1a83027`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:32:00,449::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:00,525::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0330989 s, 30.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:32:02,538::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:02,612::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0327925 s, 31.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:32:03,058::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:32:04,625::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:04,706::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383063 s, 26.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:32:06,077::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:32:06,719::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:06,793::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0325348 s, 31.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:32:07,254::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:32:07,263::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00121681 s, 3.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:32:08,806::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:08,884::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352649 s, 29.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:32:09,064::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:32:09,064::task::595::TaskManager.Task::(_updateState) Task=`c2e46f97-847a-4b5c-b932-d472277c4e8c`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:32:09,065::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:32:09,065::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:32:09,065::task::1185::TaskManager.Task::(prepare) Task=`c2e46f97-847a-4b5c-b932-d472277c4e8c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:32:09,065::task::595::TaskManager.Task::(_updateState) Task=`c2e46f97-847a-4b5c-b932-d472277c4e8c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:32:09,066::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:32:09,066::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:32:09,066::task::990::TaskManager.Task::(_decref) Task=`c2e46f97-847a-4b5c-b932-d472277c4e8c`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:32:09,071::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:32:09,071::task::595::TaskManager.Task::(_updateState) Task=`56d941a1-d9d0-427a-b5c8-7376740a4241`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:32:09,071::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:32:09,072::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f438c25d-8692-4d49-bfc6-19a75a28b32c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:32:09,072::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:32:09,072::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:32:09,072::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f438c25d-8692-4d49-bfc6-19a75a28b32c`::Granted request >Thread-338::DEBUG::2014-05-13 11:32:09,072::task::827::TaskManager.Task::(resourceAcquired) Task=`56d941a1-d9d0-427a-b5c8-7376740a4241`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:32:09,073::task::990::TaskManager.Task::(_decref) Task=`56d941a1-d9d0-427a-b5c8-7376740a4241`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:32:09,073::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:32:09,074::task::1185::TaskManager.Task::(prepare) Task=`56d941a1-d9d0-427a-b5c8-7376740a4241`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:32:09,074::task::595::TaskManager.Task::(_updateState) Task=`56d941a1-d9d0-427a-b5c8-7376740a4241`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:32:09,074::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:32:09,074::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:32:09,074::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:32:09,075::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:32:09,075::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:32:09,075::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:32:09,075::task::990::TaskManager.Task::(_decref) Task=`56d941a1-d9d0-427a-b5c8-7376740a4241`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:32:09,099::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:32:10,897::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:10,992::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0506547 s, 20.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:32:12,127::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:32:13,005::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:13,079::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310489 s, 33.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:32:15,092::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:32:15,140::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:32:15,170::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313586 s, 32.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:32:15,271::task::595::TaskManager.Task::(_updateState) Task=`95c57ea1-5466-4770-81d8-ab1f0c4b49c0`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:32:15,271::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:32:15,271::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00121681', 'lastCheck': '8.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:32:15,271::task::1185::TaskManager.Task::(prepare) Task=`95c57ea1-5466-4770-81d8-ab1f0c4b49c0`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00121681', 'lastCheck': '8.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:32:15,271::task::595::TaskManager.Task::(_updateState) Task=`95c57ea1-5466-4770-81d8-ab1f0c4b49c0`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:32:15,272::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:32:15,272::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:32:15,272::task::990::TaskManager.Task::(_decref) Task=`95c57ea1-5466-4770-81d8-ab1f0c4b49c0`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:32:17,183::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:17,262::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363988 s, 28.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:32:17,286::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:32:17,303::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000629301 s, 6.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:32:18,163::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:32:19,094::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:32:19,094::task::595::TaskManager.Task::(_updateState) Task=`4c1a0ba3-990d-4c84-9ffc-66c50e9b129d`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:32:19,095::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:32:19,095::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:32:19,095::task::1185::TaskManager.Task::(prepare) Task=`4c1a0ba3-990d-4c84-9ffc-66c50e9b129d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:32:19,095::task::595::TaskManager.Task::(_updateState) Task=`4c1a0ba3-990d-4c84-9ffc-66c50e9b129d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:32:19,096::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:32:19,096::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:32:19,096::task::990::TaskManager.Task::(_decref) Task=`4c1a0ba3-990d-4c84-9ffc-66c50e9b129d`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:32:19,102::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:32:19,103::task::595::TaskManager.Task::(_updateState) Task=`cd5f9dba-6088-4940-99ed-2971122787a4`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:32:19,103::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:32:19,103::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6cde763c-54e1-44ea-a1d3-49ab6a526d53`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:32:19,103::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:32:19,104::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:32:19,104::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6cde763c-54e1-44ea-a1d3-49ab6a526d53`::Granted request >Thread-338::DEBUG::2014-05-13 11:32:19,104::task::827::TaskManager.Task::(resourceAcquired) Task=`cd5f9dba-6088-4940-99ed-2971122787a4`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:32:19,104::task::990::TaskManager.Task::(_decref) Task=`cd5f9dba-6088-4940-99ed-2971122787a4`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:32:19,105::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:32:19,105::task::1185::TaskManager.Task::(prepare) Task=`cd5f9dba-6088-4940-99ed-2971122787a4`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:32:19,105::task::595::TaskManager.Task::(_updateState) Task=`cd5f9dba-6088-4940-99ed-2971122787a4`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:32:19,106::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:32:19,106::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:32:19,106::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:32:19,106::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:32:19,106::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:32:19,107::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:32:19,107::task::990::TaskManager.Task::(_decref) Task=`cd5f9dba-6088-4940-99ed-2971122787a4`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:32:19,326::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:19,407::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0370639 s, 27.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:32:21,183::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:32:21,420::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:21,494::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0302154 s, 33.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:32:23,507::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:23,585::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0351477 s, 29.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:32:24,208::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:32:25,598::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:25,697::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0553543 s, 18.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:32:27,234::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:32:27,323::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:32:27,331::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000629673 s, 6.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:32:27,710::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:27,788::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358551 s, 28.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:32:29,128::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6019f20] >Thread-13::DEBUG::2014-05-13 11:32:29,129::task::595::TaskManager.Task::(_updateState) Task=`efc976bd-4151-4004-8b14-a85c5d5837c2`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:32:29,129::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:32:29,130::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:32:29,130::task::1185::TaskManager.Task::(prepare) Task=`efc976bd-4151-4004-8b14-a85c5d5837c2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:32:29,130::task::595::TaskManager.Task::(_updateState) Task=`efc976bd-4151-4004-8b14-a85c5d5837c2`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:32:29,130::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:32:29,130::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:32:29,130::task::990::TaskManager.Task::(_decref) Task=`efc976bd-4151-4004-8b14-a85c5d5837c2`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:32:29,135::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6019f20] >Thread-338::DEBUG::2014-05-13 11:32:29,135::task::595::TaskManager.Task::(_updateState) Task=`dc5088ae-3527-496f-8786-afd470ba2057`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:32:29,135::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:32:29,136::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b83205c1-1199-427a-9ff6-f05a97564bcf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:32:29,136::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:32:29,136::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:32:29,136::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b83205c1-1199-427a-9ff6-f05a97564bcf`::Granted request >Thread-338::DEBUG::2014-05-13 11:32:29,137::task::827::TaskManager.Task::(resourceAcquired) Task=`dc5088ae-3527-496f-8786-afd470ba2057`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:32:29,137::task::990::TaskManager.Task::(_decref) Task=`dc5088ae-3527-496f-8786-afd470ba2057`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:32:29,138::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:32:29,138::task::1185::TaskManager.Task::(prepare) Task=`dc5088ae-3527-496f-8786-afd470ba2057`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:32:29,138::task::595::TaskManager.Task::(_updateState) Task=`dc5088ae-3527-496f-8786-afd470ba2057`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:32:29,138::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:32:29,138::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:32:29,139::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:32:29,139::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:32:29,139::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:32:29,139::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:32:29,139::task::990::TaskManager.Task::(_decref) Task=`dc5088ae-3527-496f-8786-afd470ba2057`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:32:29,801::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:29,875::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310704 s, 33.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:32:30,252::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:32:30,404::task::595::TaskManager.Task::(_updateState) Task=`87c544c0-3118-4f50-8086-d2a19372c385`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:32:30,404::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:32:30,404::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000629673', 'lastCheck': '3.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:32:30,405::task::1185::TaskManager.Task::(prepare) Task=`87c544c0-3118-4f50-8086-d2a19372c385`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000629673', 'lastCheck': '3.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:32:30,405::task::595::TaskManager.Task::(_updateState) Task=`87c544c0-3118-4f50-8086-d2a19372c385`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:32:30,405::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:32:30,405::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:32:30,405::task::990::TaskManager.Task::(_decref) Task=`87c544c0-3118-4f50-8086-d2a19372c385`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:32:31,888::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:31,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364922 s, 28.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:32:33,271::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:32:33,980::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:34,059::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357924 s, 28.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:32:36,072::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:36,146::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307601 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:32:36,291::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:32:37,339::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:32:37,348::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000635113 s, 6.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:32:38,160::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:38,238::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357777 s, 28.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:32:39,156::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:32:39,157::task::595::TaskManager.Task::(_updateState) Task=`d0cef485-27e1-46c9-93ad-fa1df78c4fc5`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:32:39,157::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:32:39,157::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:32:39,158::task::1185::TaskManager.Task::(prepare) Task=`d0cef485-27e1-46c9-93ad-fa1df78c4fc5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:32:39,158::task::595::TaskManager.Task::(_updateState) Task=`d0cef485-27e1-46c9-93ad-fa1df78c4fc5`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:32:39,158::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:32:39,158::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:32:39,158::task::990::TaskManager.Task::(_decref) Task=`d0cef485-27e1-46c9-93ad-fa1df78c4fc5`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:32:39,315::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-338::DEBUG::2014-05-13 11:32:39,433::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:32:39,433::task::595::TaskManager.Task::(_updateState) Task=`a6796793-028a-4b19-8d27-ac4975831b98`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:32:39,434::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:32:39,434::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d125c88f-47b9-479c-affb-51c8552ba624`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:32:39,434::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:32:39,434::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:32:39,435::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d125c88f-47b9-479c-affb-51c8552ba624`::Granted request >Thread-338::DEBUG::2014-05-13 11:32:39,436::task::827::TaskManager.Task::(resourceAcquired) Task=`a6796793-028a-4b19-8d27-ac4975831b98`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:32:39,437::task::990::TaskManager.Task::(_decref) Task=`a6796793-028a-4b19-8d27-ac4975831b98`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:32:39,437::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:32:39,437::task::1185::TaskManager.Task::(prepare) Task=`a6796793-028a-4b19-8d27-ac4975831b98`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:32:39,438::task::595::TaskManager.Task::(_updateState) Task=`a6796793-028a-4b19-8d27-ac4975831b98`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:32:39,438::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:32:39,438::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:32:39,438::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:32:39,438::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:32:39,439::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:32:39,439::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:32:39,439::task::990::TaskManager.Task::(_decref) Task=`a6796793-028a-4b19-8d27-ac4975831b98`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:32:40,251::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:40,617::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.324067 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:32:42,336::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:32:42,630::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:42,703::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305241 s, 33.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:32:44,716::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:44,789::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0294996 s, 34.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:32:45,354::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:32:45,545::task::595::TaskManager.Task::(_updateState) Task=`e0d9d86a-0f59-48b9-bcc3-298de18dca47`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:32:45,545::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:32:45,545::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000635113', 'lastCheck': '8.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:32:45,545::task::1185::TaskManager.Task::(prepare) Task=`e0d9d86a-0f59-48b9-bcc3-298de18dca47`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000635113', 'lastCheck': '8.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:32:45,546::task::595::TaskManager.Task::(_updateState) Task=`e0d9d86a-0f59-48b9-bcc3-298de18dca47`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:32:45,546::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:32:45,546::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:32:45,546::task::990::TaskManager.Task::(_decref) Task=`e0d9d86a-0f59-48b9-bcc3-298de18dca47`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:32:46,802::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:46,881::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362554 s, 28.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:32:47,356::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:32:47,364::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000707909 s, 5.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:32:48,538::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:32:48,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:48,973::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.034972 s, 29.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:32:49,468::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1ea398d3] >Thread-13::DEBUG::2014-05-13 11:32:49,468::task::595::TaskManager.Task::(_updateState) Task=`d065e99f-f6d5-45c2-8924-6a40dbdaebb5`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:32:49,469::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:32:49,469::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:32:49,469::task::1185::TaskManager.Task::(prepare) Task=`d065e99f-f6d5-45c2-8924-6a40dbdaebb5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:32:49,469::task::595::TaskManager.Task::(_updateState) Task=`d065e99f-f6d5-45c2-8924-6a40dbdaebb5`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:32:49,469::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:32:49,470::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:32:49,470::task::990::TaskManager.Task::(_decref) Task=`d065e99f-f6d5-45c2-8924-6a40dbdaebb5`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:32:49,478::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1ea398d3] >Thread-338::DEBUG::2014-05-13 11:32:49,479::task::595::TaskManager.Task::(_updateState) Task=`208ea65b-98a9-4c25-9008-6dce115bbaa4`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:32:49,479::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:32:49,479::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`79caf2dd-12c2-4cca-ab8d-dffb94f1760d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:32:49,480::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:32:49,480::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:32:49,480::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`79caf2dd-12c2-4cca-ab8d-dffb94f1760d`::Granted request >Thread-338::DEBUG::2014-05-13 11:32:49,480::task::827::TaskManager.Task::(resourceAcquired) Task=`208ea65b-98a9-4c25-9008-6dce115bbaa4`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:32:49,481::task::990::TaskManager.Task::(_decref) Task=`208ea65b-98a9-4c25-9008-6dce115bbaa4`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:32:49,481::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:32:49,481::task::1185::TaskManager.Task::(prepare) Task=`208ea65b-98a9-4c25-9008-6dce115bbaa4`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:32:49,482::task::595::TaskManager.Task::(_updateState) Task=`208ea65b-98a9-4c25-9008-6dce115bbaa4`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:32:49,482::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:32:49,482::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:32:49,482::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:32:49,482::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:32:49,483::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:32:49,483::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:32:49,483::task::990::TaskManager.Task::(_decref) Task=`208ea65b-98a9-4c25-9008-6dce115bbaa4`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:32:50,986::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:51,061::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309739 s, 33.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:32:51,557::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:32:53,074::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:53,451::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.321614 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:32:54,578::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:32:55,464::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:55,543::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361298 s, 28.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:32:57,371::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:32:57,379::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000713951 s, 5.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:32:57,556::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:32:57,599::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:32:57,638::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.038457 s, 26.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:32:59,503::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6623fb10] >Thread-13::DEBUG::2014-05-13 11:32:59,504::task::595::TaskManager.Task::(_updateState) Task=`207eab64-4d28-4586-a605-181ac2be2aa0`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:32:59,504::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:32:59,505::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:32:59,505::task::1185::TaskManager.Task::(prepare) Task=`207eab64-4d28-4586-a605-181ac2be2aa0`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:32:59,505::task::595::TaskManager.Task::(_updateState) Task=`207eab64-4d28-4586-a605-181ac2be2aa0`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:32:59,505::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:32:59,505::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:32:59,505::task::990::TaskManager.Task::(_decref) Task=`207eab64-4d28-4586-a605-181ac2be2aa0`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:32:59,510::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6623fb10] >Thread-338::DEBUG::2014-05-13 11:32:59,510::task::595::TaskManager.Task::(_updateState) Task=`e805322a-d4c6-4646-8002-5edbe833c39c`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:32:59,510::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:32:59,511::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f965dced-e7e8-4e05-a35b-7f574dbd0450`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:32:59,511::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:32:59,511::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:32:59,511::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f965dced-e7e8-4e05-a35b-7f574dbd0450`::Granted request >Thread-338::DEBUG::2014-05-13 11:32:59,512::task::827::TaskManager.Task::(resourceAcquired) Task=`e805322a-d4c6-4646-8002-5edbe833c39c`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:32:59,512::task::990::TaskManager.Task::(_decref) Task=`e805322a-d4c6-4646-8002-5edbe833c39c`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:32:59,512::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:32:59,513::task::1185::TaskManager.Task::(prepare) Task=`e805322a-d4c6-4646-8002-5edbe833c39c`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:32:59,513::task::595::TaskManager.Task::(_updateState) Task=`e805322a-d4c6-4646-8002-5edbe833c39c`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:32:59,513::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:32:59,513::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:32:59,513::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:32:59,513::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:32:59,514::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:32:59,514::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:32:59,514::task::990::TaskManager.Task::(_decref) Task=`e805322a-d4c6-4646-8002-5edbe833c39c`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:32:59,651::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:32:59,725::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.030963 s, 33.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:33:00,629::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:33:00,678::task::595::TaskManager.Task::(_updateState) Task=`ccd49793-7dcb-4b7f-9698-c4c48c978962`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:33:00,678::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:33:00,678::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000713951', 'lastCheck': '3.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:33:00,678::task::1185::TaskManager.Task::(prepare) Task=`ccd49793-7dcb-4b7f-9698-c4c48c978962`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000713951', 'lastCheck': '3.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:33:00,679::task::595::TaskManager.Task::(_updateState) Task=`ccd49793-7dcb-4b7f-9698-c4c48c978962`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:33:00,679::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:33:00,679::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:33:00,679::task::990::TaskManager.Task::(_decref) Task=`ccd49793-7dcb-4b7f-9698-c4c48c978962`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:33:01,738::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:01,817::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035291 s, 29.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:33:03,647::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:33:03,830::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:03,908::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0354373 s, 28.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:33:05,921::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:06,008::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0432895 s, 23.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:33:06,666::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:33:07,386::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:33:07,395::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000885855 s, 4.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:33:08,021::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:08,102::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0378106 s, 27.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:33:09,531::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:33:09,532::task::595::TaskManager.Task::(_updateState) Task=`69022fb1-e3a2-4b0e-ab80-af2a93730efe`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:33:09,532::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:33:09,533::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:33:09,533::task::1185::TaskManager.Task::(prepare) Task=`69022fb1-e3a2-4b0e-ab80-af2a93730efe`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:33:09,533::task::595::TaskManager.Task::(_updateState) Task=`69022fb1-e3a2-4b0e-ab80-af2a93730efe`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:33:09,533::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:33:09,533::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:33:09,533::task::990::TaskManager.Task::(_decref) Task=`69022fb1-e3a2-4b0e-ab80-af2a93730efe`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:33:09,539::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:33:09,539::task::595::TaskManager.Task::(_updateState) Task=`9b0dae48-0c8e-4ef5-a4f7-798f7a19557a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:33:09,539::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:33:09,540::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fd643007-5e01-42d9-89a7-5c01f721f7f0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:33:09,540::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:33:09,540::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:33:09,540::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fd643007-5e01-42d9-89a7-5c01f721f7f0`::Granted request >Thread-338::DEBUG::2014-05-13 11:33:09,541::task::827::TaskManager.Task::(resourceAcquired) Task=`9b0dae48-0c8e-4ef5-a4f7-798f7a19557a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:33:09,541::task::990::TaskManager.Task::(_decref) Task=`9b0dae48-0c8e-4ef5-a4f7-798f7a19557a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:33:09,542::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:33:09,542::task::1185::TaskManager.Task::(prepare) Task=`9b0dae48-0c8e-4ef5-a4f7-798f7a19557a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:33:09,542::task::595::TaskManager.Task::(_updateState) Task=`9b0dae48-0c8e-4ef5-a4f7-798f7a19557a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:33:09,542::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:33:09,542::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:33:09,543::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:33:09,543::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:33:09,543::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:33:09,543::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:33:09,543::task::990::TaskManager.Task::(_decref) Task=`9b0dae48-0c8e-4ef5-a4f7-798f7a19557a`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:33:09,686::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:33:10,115::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:10,204::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0456353 s, 22.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:33:12,217::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:12,310::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307124 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:33:12,708::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:33:14,499::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:14,573::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306379 s, 33.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:33:15,728::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:33:15,813::task::595::TaskManager.Task::(_updateState) Task=`b21853ea-f8c0-47f5-a87a-46a0333198e5`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:33:15,813::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:33:15,813::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000885855', 'lastCheck': '8.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:33:15,813::task::1185::TaskManager.Task::(prepare) Task=`b21853ea-f8c0-47f5-a87a-46a0333198e5`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000885855', 'lastCheck': '8.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:33:15,814::task::595::TaskManager.Task::(_updateState) Task=`b21853ea-f8c0-47f5-a87a-46a0333198e5`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:33:15,814::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:33:15,814::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:33:15,814::task::990::TaskManager.Task::(_decref) Task=`b21853ea-f8c0-47f5-a87a-46a0333198e5`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:33:16,586::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:16,667::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0376255 s, 27.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:33:17,402::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:33:17,411::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000670796 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:33:18,679::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:33:18,750::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:33:18,763::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0372671 s, 27.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:33:19,564::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:33:19,564::task::595::TaskManager.Task::(_updateState) Task=`0d7d02c0-d21c-4861-906d-9ceffbac9c22`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:33:19,564::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:33:19,565::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:33:19,565::task::1185::TaskManager.Task::(prepare) Task=`0d7d02c0-d21c-4861-906d-9ceffbac9c22`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:33:19,565::task::595::TaskManager.Task::(_updateState) Task=`0d7d02c0-d21c-4861-906d-9ceffbac9c22`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:33:19,565::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:33:19,565::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:33:19,566::task::990::TaskManager.Task::(_decref) Task=`0d7d02c0-d21c-4861-906d-9ceffbac9c22`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:33:19,570::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:33:19,571::task::595::TaskManager.Task::(_updateState) Task=`388e9452-c7a7-462a-ac14-0be6c38fe580`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:33:19,571::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:33:19,571::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`84baa7e2-9bf6-4b89-b03b-f4e43d5b5bf0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:33:19,571::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:33:19,572::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:33:19,572::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`84baa7e2-9bf6-4b89-b03b-f4e43d5b5bf0`::Granted request >Thread-338::DEBUG::2014-05-13 11:33:19,572::task::827::TaskManager.Task::(resourceAcquired) Task=`388e9452-c7a7-462a-ac14-0be6c38fe580`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:33:19,572::task::990::TaskManager.Task::(_decref) Task=`388e9452-c7a7-462a-ac14-0be6c38fe580`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:33:19,573::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:33:19,573::task::1185::TaskManager.Task::(prepare) Task=`388e9452-c7a7-462a-ac14-0be6c38fe580`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:33:19,573::task::595::TaskManager.Task::(_updateState) Task=`388e9452-c7a7-462a-ac14-0be6c38fe580`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:33:19,574::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:33:19,574::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:33:19,574::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:33:19,574::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:33:19,574::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:33:19,575::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:33:19,575::task::990::TaskManager.Task::(_decref) Task=`388e9452-c7a7-462a-ac14-0be6c38fe580`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:33:20,775::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:21,137::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.319382 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:33:21,770::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:33:23,150::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:23,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0348497 s, 29.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:33:24,789::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:33:25,242::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:25,319::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0342462 s, 29.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:33:27,332::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:27,408::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321752 s, 31.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:33:27,434::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:33:27,452::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00079097 s, 5.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:33:27,808::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:33:29,477::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:29,560::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362515 s, 28.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:33:29,592::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:33:29,592::task::595::TaskManager.Task::(_updateState) Task=`0b035e1b-eed9-4442-8907-716bc69e2125`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:33:29,593::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:33:29,593::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:33:29,593::task::1185::TaskManager.Task::(prepare) Task=`0b035e1b-eed9-4442-8907-716bc69e2125`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:33:29,593::task::595::TaskManager.Task::(_updateState) Task=`0b035e1b-eed9-4442-8907-716bc69e2125`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:33:29,593::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:33:29,594::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:33:29,594::task::990::TaskManager.Task::(_decref) Task=`0b035e1b-eed9-4442-8907-716bc69e2125`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:33:29,598::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:33:29,599::task::595::TaskManager.Task::(_updateState) Task=`73001a90-b98c-47be-8bd5-b72d463247ba`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:33:29,599::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:33:29,599::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`62f593a3-ac57-4aef-b5a2-d4dfc11788be`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:33:29,599::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:33:29,600::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:33:29,600::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`62f593a3-ac57-4aef-b5a2-d4dfc11788be`::Granted request >Thread-338::DEBUG::2014-05-13 11:33:29,600::task::827::TaskManager.Task::(resourceAcquired) Task=`73001a90-b98c-47be-8bd5-b72d463247ba`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:33:29,600::task::990::TaskManager.Task::(_decref) Task=`73001a90-b98c-47be-8bd5-b72d463247ba`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:33:29,601::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:33:29,601::task::1185::TaskManager.Task::(prepare) Task=`73001a90-b98c-47be-8bd5-b72d463247ba`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:33:29,601::task::595::TaskManager.Task::(_updateState) Task=`73001a90-b98c-47be-8bd5-b72d463247ba`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:33:29,602::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:33:29,602::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:33:29,602::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:33:29,602::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:33:29,602::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:33:29,603::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:33:29,603::task::990::TaskManager.Task::(_decref) Task=`73001a90-b98c-47be-8bd5-b72d463247ba`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:33:30,826::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:33:30,942::task::595::TaskManager.Task::(_updateState) Task=`4d733ca3-d7c8-4806-927a-7410766d4015`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:33:30,942::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:33:30,942::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00079097', 'lastCheck': '3.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:33:30,942::task::1185::TaskManager.Task::(prepare) Task=`4d733ca3-d7c8-4806-927a-7410766d4015`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00079097', 'lastCheck': '3.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:33:30,942::task::595::TaskManager.Task::(_updateState) Task=`4d733ca3-d7c8-4806-927a-7410766d4015`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:33:30,942::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:33:30,943::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:33:30,943::task::990::TaskManager.Task::(_decref) Task=`4d733ca3-d7c8-4806-927a-7410766d4015`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:33:31,573::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:31,664::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0484452 s, 21.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:33:33,677::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:33,762::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0336242 s, 30.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:33:33,849::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:33:35,775::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:35,848::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0301373 s, 34.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:33:36,873::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:33:37,476::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:33:37,486::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00169426 s, 2.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:33:37,861::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:37,939::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0348609 s, 29.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:33:39,622::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:33:39,623::task::595::TaskManager.Task::(_updateState) Task=`d73355e9-5bcd-4c62-a32e-2f736908a4bf`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:33:39,623::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:33:39,624::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:33:39,624::task::1185::TaskManager.Task::(prepare) Task=`d73355e9-5bcd-4c62-a32e-2f736908a4bf`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:33:39,624::task::595::TaskManager.Task::(_updateState) Task=`d73355e9-5bcd-4c62-a32e-2f736908a4bf`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:33:39,624::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:33:39,624::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:33:39,624::task::990::TaskManager.Task::(_decref) Task=`d73355e9-5bcd-4c62-a32e-2f736908a4bf`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:33:39,629::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:33:39,629::task::595::TaskManager.Task::(_updateState) Task=`1ec8f21a-f9d8-45f1-a94a-5ad680848815`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:33:39,630::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:33:39,630::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1350bacf-15ff-40c7-9a32-99fe76eff7b4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:33:39,630::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:33:39,630::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:33:39,631::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1350bacf-15ff-40c7-9a32-99fe76eff7b4`::Granted request >Thread-338::DEBUG::2014-05-13 11:33:39,631::task::827::TaskManager.Task::(resourceAcquired) Task=`1ec8f21a-f9d8-45f1-a94a-5ad680848815`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:33:39,631::task::990::TaskManager.Task::(_decref) Task=`1ec8f21a-f9d8-45f1-a94a-5ad680848815`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:33:39,632::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:33:39,632::task::1185::TaskManager.Task::(prepare) Task=`1ec8f21a-f9d8-45f1-a94a-5ad680848815`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:33:39,632::task::595::TaskManager.Task::(_updateState) Task=`1ec8f21a-f9d8-45f1-a94a-5ad680848815`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:33:39,632::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:33:39,633::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:33:39,633::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:33:39,633::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:33:39,633::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:33:39,633::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:33:39,634::task::990::TaskManager.Task::(_decref) Task=`1ec8f21a-f9d8-45f1-a94a-5ad680848815`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:33:39,896::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:33:39,952::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:40,031::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373782 s, 27.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:33:42,044::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:42,120::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0335736 s, 30.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:33:42,915::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:33:44,133::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:44,212::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356424 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:33:45,933::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:33:46,076::task::595::TaskManager.Task::(_updateState) Task=`bc88057d-a4e9-44c8-92ce-557ddb993053`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:33:46,076::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:33:46,076::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00169426', 'lastCheck': '8.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:33:46,076::task::1185::TaskManager.Task::(prepare) Task=`bc88057d-a4e9-44c8-92ce-557ddb993053`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00169426', 'lastCheck': '8.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:33:46,076::task::595::TaskManager.Task::(_updateState) Task=`bc88057d-a4e9-44c8-92ce-557ddb993053`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:33:46,077::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:33:46,077::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:33:46,077::task::990::TaskManager.Task::(_decref) Task=`bc88057d-a4e9-44c8-92ce-557ddb993053`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:33:46,225::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:46,297::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0308005 s, 33.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:33:47,492::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:33:47,502::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00167119 s, 2.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:33:48,310::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:48,405::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0505283 s, 20.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:33:48,957::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:33:49,654::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:33:49,655::task::595::TaskManager.Task::(_updateState) Task=`051e5ca5-ddbb-41f5-9899-319a15a220b5`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:33:49,655::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:33:49,655::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:33:49,656::task::1185::TaskManager.Task::(prepare) Task=`051e5ca5-ddbb-41f5-9899-319a15a220b5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:33:49,656::task::595::TaskManager.Task::(_updateState) Task=`051e5ca5-ddbb-41f5-9899-319a15a220b5`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:33:49,656::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:33:49,656::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:33:49,656::task::990::TaskManager.Task::(_decref) Task=`051e5ca5-ddbb-41f5-9899-319a15a220b5`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:33:49,663::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:33:49,663::task::595::TaskManager.Task::(_updateState) Task=`3d4efb4f-669a-44a6-a6d1-bd117fc7110f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:33:49,663::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:33:49,665::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3c346b65-f889-43c4-a12f-a3c9412cd213`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:33:49,665::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:33:49,665::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:33:49,665::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3c346b65-f889-43c4-a12f-a3c9412cd213`::Granted request >Thread-338::DEBUG::2014-05-13 11:33:49,666::task::827::TaskManager.Task::(resourceAcquired) Task=`3d4efb4f-669a-44a6-a6d1-bd117fc7110f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:33:49,666::task::990::TaskManager.Task::(_decref) Task=`3d4efb4f-669a-44a6-a6d1-bd117fc7110f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:33:49,667::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:33:49,667::task::1185::TaskManager.Task::(prepare) Task=`3d4efb4f-669a-44a6-a6d1-bd117fc7110f`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:33:49,667::task::595::TaskManager.Task::(_updateState) Task=`3d4efb4f-669a-44a6-a6d1-bd117fc7110f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:33:49,667::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:33:49,667::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:33:49,668::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:33:49,668::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:33:49,668::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:33:49,668::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:33:49,668::task::990::TaskManager.Task::(_decref) Task=`3d4efb4f-669a-44a6-a6d1-bd117fc7110f`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:33:50,419::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:50,493::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306697 s, 33.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:33:51,979::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:33:52,506::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:52,585::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0345886 s, 29.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:33:54,695::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:54,773::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.034684 s, 29.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:33:54,997::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:33:56,786::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:56,860::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0300267 s, 34.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:33:57,509::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:33:57,517::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000639756 s, 6.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:33:58,021::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:33:58,873::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:33:58,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0348896 s, 29.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:33:59,687::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:33:59,688::task::595::TaskManager.Task::(_updateState) Task=`db71805f-1802-4fc5-bfc7-7741c9d69b7f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:33:59,688::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:33:59,688::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:33:59,688::task::1185::TaskManager.Task::(prepare) Task=`db71805f-1802-4fc5-bfc7-7741c9d69b7f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:33:59,688::task::595::TaskManager.Task::(_updateState) Task=`db71805f-1802-4fc5-bfc7-7741c9d69b7f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:33:59,689::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:33:59,689::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:33:59,689::task::990::TaskManager.Task::(_decref) Task=`db71805f-1802-4fc5-bfc7-7741c9d69b7f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:33:59,694::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:33:59,694::task::595::TaskManager.Task::(_updateState) Task=`cf214452-25d1-49df-ac26-7fccb68d50b2`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:33:59,694::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:33:59,695::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`77f558e7-5b1c-4efc-a4cd-8d2a06bb2530`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:33:59,695::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:33:59,695::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:33:59,695::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`77f558e7-5b1c-4efc-a4cd-8d2a06bb2530`::Granted request >Thread-338::DEBUG::2014-05-13 11:33:59,696::task::827::TaskManager.Task::(resourceAcquired) Task=`cf214452-25d1-49df-ac26-7fccb68d50b2`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:33:59,696::task::990::TaskManager.Task::(_decref) Task=`cf214452-25d1-49df-ac26-7fccb68d50b2`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:33:59,696::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:33:59,697::task::1185::TaskManager.Task::(prepare) Task=`cf214452-25d1-49df-ac26-7fccb68d50b2`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:33:59,697::task::595::TaskManager.Task::(_updateState) Task=`cf214452-25d1-49df-ac26-7fccb68d50b2`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:33:59,697::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:33:59,697::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:33:59,697::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:33:59,697::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:33:59,698::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:33:59,698::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:33:59,698::task::990::TaskManager.Task::(_decref) Task=`cf214452-25d1-49df-ac26-7fccb68d50b2`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:34:00,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:34:01,040::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:34:01,221::task::595::TaskManager.Task::(_updateState) Task=`ad150087-fcff-47a5-9f1a-37fb26475c4b`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:34:01,221::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:34:01,221::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000639756', 'lastCheck': '3.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:34:01,221::task::1185::TaskManager.Task::(prepare) Task=`ad150087-fcff-47a5-9f1a-37fb26475c4b`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000639756', 'lastCheck': '3.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:34:01,221::task::595::TaskManager.Task::(_updateState) Task=`ad150087-fcff-47a5-9f1a-37fb26475c4b`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:34:01,221::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:34:01,222::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:34:01,222::task::990::TaskManager.Task::(_decref) Task=`ad150087-fcff-47a5-9f1a-37fb26475c4b`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:34:01,332::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.32526 s, 3.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:34:03,345::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:34:03,419::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0296406 s, 34.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:34:04,066::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:34:05,432::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:34:05,505::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0299297 s, 34.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:34:07,109::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:34:07,519::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 11:34:07,532::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:34:07,550::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000673241 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:34:07,646::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0719507 s, 14.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:34:09,659::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 11:34:09,719::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:34:09,720::task::595::TaskManager.Task::(_updateState) Task=`fdb37701-c7f3-44e2-9389-8f4ce1f4e6c6`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:34:09,722::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:34:09,723::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:34:09,724::task::1185::TaskManager.Task::(prepare) Task=`fdb37701-c7f3-44e2-9389-8f4ce1f4e6c6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:34:09,725::task::595::TaskManager.Task::(_updateState) Task=`fdb37701-c7f3-44e2-9389-8f4ce1f4e6c6`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:34:09,726::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:34:09,727::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:34:09,728::task::990::TaskManager.Task::(_decref) Task=`fdb37701-c7f3-44e2-9389-8f4ce1f4e6c6`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:34:09,736::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:34:09,738::task::595::TaskManager.Task::(_updateState) Task=`8cd4e2be-39c4-4ad2-923f-fe0903e1a0a3`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:34:09,739::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:34:09,741::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0d08971b-bb30-4585-a68e-e678d05db0fd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:34:09,742::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:34:09,743::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:34:09,744::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0d08971b-bb30-4585-a68e-e678d05db0fd`::Granted request >Thread-338::DEBUG::2014-05-13 11:34:09,746::task::827::TaskManager.Task::(resourceAcquired) Task=`8cd4e2be-39c4-4ad2-923f-fe0903e1a0a3`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:34:09,747::task::990::TaskManager.Task::(_decref) Task=`8cd4e2be-39c4-4ad2-923f-fe0903e1a0a3`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:34:09,751::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:34:09,751::task::1185::TaskManager.Task::(prepare) Task=`8cd4e2be-39c4-4ad2-923f-fe0903e1a0a3`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:34:09,752::task::595::TaskManager.Task::(_updateState) Task=`8cd4e2be-39c4-4ad2-923f-fe0903e1a0a3`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:34:09,753::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:34:09,754::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:34:09,755::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:34:09,756::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:34:09,757::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:34:09,758::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:34:09,759::task::990::TaskManager.Task::(_decref) Task=`8cd4e2be-39c4-4ad2-923f-fe0903e1a0a3`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:34:09,776::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356863 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:34:10,127::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:34:11,789::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:34:11,863::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307998 s, 33.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:34:13,147::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:34:13,876::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:34:14,237::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.318219 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:34:16,169::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:34:16,250::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:34:16,324::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323904 s, 31.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:34:16,355::task::595::TaskManager.Task::(_updateState) Task=`220b0ed0-3595-4aa2-a6d4-954b1602e4ba`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:34:16,355::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:34:16,355::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000673241', 'lastCheck': '8.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:34:16,355::task::1185::TaskManager.Task::(prepare) Task=`220b0ed0-3595-4aa2-a6d4-954b1602e4ba`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000673241', 'lastCheck': '8.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:34:16,356::task::595::TaskManager.Task::(_updateState) Task=`220b0ed0-3595-4aa2-a6d4-954b1602e4ba`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:34:16,356::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:34:16,356::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:34:16,356::task::990::TaskManager.Task::(_decref) Task=`220b0ed0-3595-4aa2-a6d4-954b1602e4ba`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 11:34:17,576::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:34:17,584::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000700745 s, 5.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:34:18,337::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:34:18,418::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0375729 s, 27.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:34:19,187::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:34:19,788::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [19876d15] >Thread-13::DEBUG::2014-05-13 11:34:19,789::task::595::TaskManager.Task::(_updateState) Task=`4789882f-2e5c-4fba-a625-de3615e73218`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:34:19,789::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:34:19,789::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:34:19,790::task::1185::TaskManager.Task::(prepare) Task=`4789882f-2e5c-4fba-a625-de3615e73218`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:34:19,790::task::595::TaskManager.Task::(_updateState) Task=`4789882f-2e5c-4fba-a625-de3615e73218`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:34:19,790::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:34:19,790::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:34:19,790::task::990::TaskManager.Task::(_decref) Task=`4789882f-2e5c-4fba-a625-de3615e73218`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:34:19,795::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [19876d15] >Thread-338::DEBUG::2014-05-13 11:34:19,795::task::595::TaskManager.Task::(_updateState) Task=`676250ff-a191-4dec-a2c9-8bf8695719b1`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:34:19,795::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:34:19,796::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c6edf918-6617-4f4e-ad89-66a9722b6eac`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:34:19,796::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:34:19,796::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:34:19,796::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c6edf918-6617-4f4e-ad89-66a9722b6eac`::Granted request >Thread-338::DEBUG::2014-05-13 11:34:19,797::task::827::TaskManager.Task::(resourceAcquired) Task=`676250ff-a191-4dec-a2c9-8bf8695719b1`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:34:19,797::task::990::TaskManager.Task::(_decref) Task=`676250ff-a191-4dec-a2c9-8bf8695719b1`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:34:19,798::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:34:19,798::task::1185::TaskManager.Task::(prepare) Task=`676250ff-a191-4dec-a2c9-8bf8695719b1`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:34:19,798::task::595::TaskManager.Task::(_updateState) Task=`676250ff-a191-4dec-a2c9-8bf8695719b1`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:34:19,798::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:34:19,798::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:34:19,799::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:34:19,799::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:34:19,799::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:34:19,799::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:34:19,799::task::990::TaskManager.Task::(_decref) Task=`676250ff-a191-4dec-a2c9-8bf8695719b1`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:34:20,431::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:34:20,508::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0347298 s, 29.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:34:22,206::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:34:22,521::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:34:22,600::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0370102 s, 27.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:34:24,613::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:34:24,694::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0372277 s, 27.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:34:25,224::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:34:26,706::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:34:26,781::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0328808 s, 31.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:34:27,591::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:34:27,600::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000702374 s, 5.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:34:28,242::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:34:28,794::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:34:29,156::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.319152 s, 3.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:34:29,824::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:34:29,825::task::595::TaskManager.Task::(_updateState) Task=`4df2d24d-3c75-482c-b459-a93c9e250bde`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:34:29,825::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:34:29,825::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:34:29,825::task::1185::TaskManager.Task::(prepare) Task=`4df2d24d-3c75-482c-b459-a93c9e250bde`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:34:29,825::task::595::TaskManager.Task::(_updateState) Task=`4df2d24d-3c75-482c-b459-a93c9e250bde`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:34:29,826::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:34:29,826::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:34:29,826::task::990::TaskManager.Task::(_decref) Task=`4df2d24d-3c75-482c-b459-a93c9e250bde`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:34:29,832::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:34:29,832::task::595::TaskManager.Task::(_updateState) Task=`076b03e7-39c0-4b9f-8450-128f7f500dbe`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:34:29,832::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:34:29,833::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`7c3324c0-60a1-4b36-aa8b-63d95d7450d0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:34:29,833::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:34:29,833::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:34:29,833::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`7c3324c0-60a1-4b36-aa8b-63d95d7450d0`::Granted request >Thread-338::DEBUG::2014-05-13 11:34:29,834::task::827::TaskManager.Task::(resourceAcquired) Task=`076b03e7-39c0-4b9f-8450-128f7f500dbe`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:34:29,834::task::990::TaskManager.Task::(_decref) Task=`076b03e7-39c0-4b9f-8450-128f7f500dbe`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:34:29,834::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:34:29,834::task::1185::TaskManager.Task::(prepare) Task=`076b03e7-39c0-4b9f-8450-128f7f500dbe`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:34:29,835::task::595::TaskManager.Task::(_updateState) Task=`076b03e7-39c0-4b9f-8450-128f7f500dbe`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:34:29,835::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:34:29,835::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:34:29,835::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:34:29,835::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:34:29,835::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:34:29,836::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:34:29,836::task::990::TaskManager.Task::(_decref) Task=`076b03e7-39c0-4b9f-8450-128f7f500dbe`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:34:31,169::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:34:31,243::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303052 s, 33.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:34:31,260::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:34:31,485::task::595::TaskManager.Task::(_updateState) Task=`fd9163cb-fff1-455f-b32a-5d3f5fd1c037`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:34:31,486::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:34:31,486::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000702374', 'lastCheck': '3.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:34:31,486::task::1185::TaskManager.Task::(prepare) Task=`fd9163cb-fff1-455f-b32a-5d3f5fd1c037`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000702374', 'lastCheck': '3.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:34:31,486::task::595::TaskManager.Task::(_updateState) Task=`fd9163cb-fff1-455f-b32a-5d3f5fd1c037`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:34:31,487::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:34:31,487::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:34:31,487::task::990::TaskManager.Task::(_decref) Task=`fd9163cb-fff1-455f-b32a-5d3f5fd1c037`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:34:33,256::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:34:33,329::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303877 s, 33.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:34:34,281::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:34:35,342::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:34:35,426::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0399064 s, 25.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:34:37,300::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:34:37,439::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:34:37,513::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303561 s, 33.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:34:37,607::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:34:37,616::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000660631 s, 6.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:34:39,526::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:34:39,607::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0380082 s, 26.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:34:39,859::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5affc0d0] >Thread-13::DEBUG::2014-05-13 11:34:39,859::task::595::TaskManager.Task::(_updateState) Task=`f457b38b-23d5-4058-9158-30e26dbed17a`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:34:39,859::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:34:39,860::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:34:39,860::task::1185::TaskManager.Task::(prepare) Task=`f457b38b-23d5-4058-9158-30e26dbed17a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:34:39,860::task::595::TaskManager.Task::(_updateState) Task=`f457b38b-23d5-4058-9158-30e26dbed17a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:34:39,860::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:34:39,860::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:34:39,861::task::990::TaskManager.Task::(_decref) Task=`f457b38b-23d5-4058-9158-30e26dbed17a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:34:39,865::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5affc0d0] >Thread-338::DEBUG::2014-05-13 11:34:39,865::task::595::TaskManager.Task::(_updateState) Task=`2e7a0507-8be0-4cd1-9238-889d481cc592`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:34:39,865::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:34:39,866::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4d709fe4-11bc-4f1c-a3eb-1ebc1dbf8d47`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:34:39,866::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:34:39,866::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:34:39,866::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4d709fe4-11bc-4f1c-a3eb-1ebc1dbf8d47`::Granted request >Thread-338::DEBUG::2014-05-13 11:34:39,867::task::827::TaskManager.Task::(resourceAcquired) Task=`2e7a0507-8be0-4cd1-9238-889d481cc592`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:34:39,867::task::990::TaskManager.Task::(_decref) Task=`2e7a0507-8be0-4cd1-9238-889d481cc592`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:34:39,868::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:34:39,868::task::1185::TaskManager.Task::(prepare) Task=`2e7a0507-8be0-4cd1-9238-889d481cc592`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:34:39,868::task::595::TaskManager.Task::(_updateState) Task=`2e7a0507-8be0-4cd1-9238-889d481cc592`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:34:39,868::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:34:39,868::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:34:39,869::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:34:39,869::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:34:39,869::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:34:39,869::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:34:39,869::task::990::TaskManager.Task::(_decref) Task=`2e7a0507-8be0-4cd1-9238-889d481cc592`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:34:40,324::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:34:41,620::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:34:41,719::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0436363 s, 23.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:34:43,344::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:34:43,733::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:34:43,812::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356526 s, 28.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:34:45,825::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:34:45,900::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320464 s, 32.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:34:46,363::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:34:46,617::task::595::TaskManager.Task::(_updateState) Task=`e0b18549-64a2-45f6-b876-7b902b727e0b`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:34:46,617::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:34:46,617::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000660631', 'lastCheck': '9.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:34:46,617::task::1185::TaskManager.Task::(prepare) Task=`e0b18549-64a2-45f6-b876-7b902b727e0b`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000660631', 'lastCheck': '9.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:34:46,618::task::595::TaskManager.Task::(_updateState) Task=`e0b18549-64a2-45f6-b876-7b902b727e0b`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:34:46,618::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:34:46,618::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:34:46,618::task::990::TaskManager.Task::(_decref) Task=`e0b18549-64a2-45f6-b876-7b902b727e0b`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 11:34:47,622::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:34:47,631::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000698955 s, 5.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:34:47,913::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:34:47,994::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365175 s, 28.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:34:49,392::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:34:49,889::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:34:49,890::task::595::TaskManager.Task::(_updateState) Task=`d166ea50-a87f-4112-82c9-cb5e754670da`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:34:49,890::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:34:49,890::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:34:49,890::task::1185::TaskManager.Task::(prepare) Task=`d166ea50-a87f-4112-82c9-cb5e754670da`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:34:49,891::task::595::TaskManager.Task::(_updateState) Task=`d166ea50-a87f-4112-82c9-cb5e754670da`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:34:49,891::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:34:49,891::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:34:49,891::task::990::TaskManager.Task::(_decref) Task=`d166ea50-a87f-4112-82c9-cb5e754670da`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:34:49,897::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:34:49,897::task::595::TaskManager.Task::(_updateState) Task=`64fe770a-b8a6-4e4f-b5a5-54c5daaa71e7`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:34:49,897::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:34:49,898::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`aa9fbc34-d47e-49b6-a08e-2ec25d91a22b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:34:49,898::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:34:49,898::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:34:49,898::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`aa9fbc34-d47e-49b6-a08e-2ec25d91a22b`::Granted request >Thread-338::DEBUG::2014-05-13 11:34:49,899::task::827::TaskManager.Task::(resourceAcquired) Task=`64fe770a-b8a6-4e4f-b5a5-54c5daaa71e7`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:34:49,899::task::990::TaskManager.Task::(_decref) Task=`64fe770a-b8a6-4e4f-b5a5-54c5daaa71e7`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:34:49,900::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:34:49,900::task::1185::TaskManager.Task::(prepare) Task=`64fe770a-b8a6-4e4f-b5a5-54c5daaa71e7`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:34:49,900::task::595::TaskManager.Task::(_updateState) Task=`64fe770a-b8a6-4e4f-b5a5-54c5daaa71e7`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:34:49,900::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:34:49,900::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:34:49,900::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:34:49,901::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:34:49,901::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:34:49,901::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:34:49,901::task::990::TaskManager.Task::(_decref) Task=`64fe770a-b8a6-4e4f-b5a5-54c5daaa71e7`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:34:50,007::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:34:50,086::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358622 s, 28.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:34:52,099::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:34:52,173::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.030843 s, 33.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:34:52,413::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:34:54,186::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:34:54,280::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0504914 s, 20.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:34:55,433::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:34:56,292::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:34:56,377::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0417513 s, 24.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:34:57,636::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 41830dad-f012-4b4d-b636-863f66d5ca4b >Thread-322::DEBUG::2014-05-13 11:34:57,637::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-322::DEBUG::2014-05-13 11:34:57,639::blockSD::336::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:34:57,647::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000700737 s, 2.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:34:57,648::misc::287::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000700737 s, 2.9 MB/s'], size: 2048 >Thread-322::DEBUG::2014-05-13 11:34:57,648::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-322::WARNING::2014-05-13 11:34:57,648::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-322::DEBUG::2014-05-13 11:34:57,648::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-322::DEBUG::2014-05-13 11:34:57,649::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:34:57,649::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:34:57,649::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:34:57,650::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:34:57,650::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:34:57,651::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 11:34:57,891::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:34:57,892::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:34:57,893::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'LOCKPOLICY=', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', 'DESCRIPTION=ISCSI', 'LOGBLKSIZE=512', 'TYPE=ISCSI', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'PHYBLKSIZE=512', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', 'CLASS=Data', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'POOL_DESCRIPTION=DC_2', 'POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'MASTER_VERSION=1', 'ROLE=Master', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', '_SHA_CKSUM=28bb2d7cc60e0d779eb69fe93d050c16c769f069', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0'] >Thread-322::INFO::2014-05-13 11:34:57,894::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_imageNS already registered >Thread-322::INFO::2014-05-13 11:34:57,894::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_volumeNS already registered >Thread-322::INFO::2014-05-13 11:34:57,894::blockSD::452::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_lvmActivationNS already registered >Thread-322::DEBUG::2014-05-13 11:34:57,895::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 11:34:58,104::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:34:58,107::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:34:58,116::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000492174 s, 8.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:34:58,390::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:34:58,457::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:34:58,474::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0378643 s, 27.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:34:59,922::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:34:59,922::task::595::TaskManager.Task::(_updateState) Task=`e945a909-0055-43d8-8d82-fd1d2c61b6d5`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:34:59,923::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:34:59,923::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:34:59,923::task::1185::TaskManager.Task::(prepare) Task=`e945a909-0055-43d8-8d82-fd1d2c61b6d5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:34:59,923::task::595::TaskManager.Task::(_updateState) Task=`e945a909-0055-43d8-8d82-fd1d2c61b6d5`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:34:59,923::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:34:59,924::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:34:59,924::task::990::TaskManager.Task::(_decref) Task=`e945a909-0055-43d8-8d82-fd1d2c61b6d5`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:34:59,929::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:34:59,929::task::595::TaskManager.Task::(_updateState) Task=`98aab558-ca34-49f1-b176-03afb6a29244`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:34:59,929::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:34:59,930::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`203ae959-aef4-4577-98c1-2f398701f014`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:34:59,930::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:34:59,930::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:34:59,930::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`203ae959-aef4-4577-98c1-2f398701f014`::Granted request >Thread-338::DEBUG::2014-05-13 11:34:59,931::task::827::TaskManager.Task::(resourceAcquired) Task=`98aab558-ca34-49f1-b176-03afb6a29244`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:34:59,931::task::990::TaskManager.Task::(_decref) Task=`98aab558-ca34-49f1-b176-03afb6a29244`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:34:59,931::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:34:59,931::task::1185::TaskManager.Task::(prepare) Task=`98aab558-ca34-49f1-b176-03afb6a29244`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:34:59,932::task::595::TaskManager.Task::(_updateState) Task=`98aab558-ca34-49f1-b176-03afb6a29244`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:34:59,932::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:34:59,932::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:34:59,932::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:34:59,932::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:34:59,932::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:34:59,933::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:34:59,933::task::990::TaskManager.Task::(_decref) Task=`98aab558-ca34-49f1-b176-03afb6a29244`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:35:00,486::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:00,564::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352745 s, 29.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:35:01,475::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:35:01,764::task::595::TaskManager.Task::(_updateState) Task=`b247a12f-8a0d-41c9-94aa-dbe5e5d1fbec`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:35:01,765::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:35:01,767::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000492174', 'lastCheck': '3.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:35:01,768::task::1185::TaskManager.Task::(prepare) Task=`b247a12f-8a0d-41c9-94aa-dbe5e5d1fbec`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000492174', 'lastCheck': '3.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:35:01,769::task::595::TaskManager.Task::(_updateState) Task=`b247a12f-8a0d-41c9-94aa-dbe5e5d1fbec`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:35:01,770::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:35:01,771::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:35:01,772::task::990::TaskManager.Task::(_decref) Task=`b247a12f-8a0d-41c9-94aa-dbe5e5d1fbec`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:35:02,577::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:02,652::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305095 s, 33.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:35:04,494::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:35:04,665::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:04,745::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362179 s, 28.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:35:06,758::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:06,833::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313635 s, 32.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:35:07,514::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:35:08,123::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:35:08,131::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000642887 s, 6.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:35:08,845::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:09,005::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.114963 s, 8.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:35:09,954::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [12ac301a] >Thread-13::DEBUG::2014-05-13 11:35:09,955::task::595::TaskManager.Task::(_updateState) Task=`8de31505-4dbb-41b5-ac30-927bd98db572`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:35:09,955::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:35:09,955::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:35:09,955::task::1185::TaskManager.Task::(prepare) Task=`8de31505-4dbb-41b5-ac30-927bd98db572`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:35:09,955::task::595::TaskManager.Task::(_updateState) Task=`8de31505-4dbb-41b5-ac30-927bd98db572`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:35:09,956::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:35:09,956::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:35:09,956::task::990::TaskManager.Task::(_decref) Task=`8de31505-4dbb-41b5-ac30-927bd98db572`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:35:09,961::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [12ac301a] >Thread-338::DEBUG::2014-05-13 11:35:09,961::task::595::TaskManager.Task::(_updateState) Task=`7a607034-5dfa-4142-af01-68af4009fbd4`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:35:09,961::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:35:09,962::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0dcde3b7-6d71-4386-bbed-7aa25e476489`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:35:09,962::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:35:09,962::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:35:09,962::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0dcde3b7-6d71-4386-bbed-7aa25e476489`::Granted request >Thread-338::DEBUG::2014-05-13 11:35:09,963::task::827::TaskManager.Task::(resourceAcquired) Task=`7a607034-5dfa-4142-af01-68af4009fbd4`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:35:09,963::task::990::TaskManager.Task::(_decref) Task=`7a607034-5dfa-4142-af01-68af4009fbd4`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:35:09,964::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:35:09,964::task::1185::TaskManager.Task::(prepare) Task=`7a607034-5dfa-4142-af01-68af4009fbd4`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:35:09,964::task::595::TaskManager.Task::(_updateState) Task=`7a607034-5dfa-4142-af01-68af4009fbd4`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:35:09,964::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:35:09,964::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:35:09,965::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:35:09,965::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:35:09,965::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:35:09,965::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:35:09,965::task::990::TaskManager.Task::(_decref) Task=`7a607034-5dfa-4142-af01-68af4009fbd4`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:35:10,531::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:35:11,018::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:11,096::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0354613 s, 28.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:35:13,109::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:13,184::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317141 s, 32.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:35:13,550::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:35:15,197::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:15,271::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.030619 s, 33.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:35:16,576::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:35:16,912::task::595::TaskManager.Task::(_updateState) Task=`a6085ac8-3dc7-4bea-9ce9-2286f9030dee`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:35:16,912::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:35:16,912::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000642887', 'lastCheck': '8.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:35:16,913::task::1185::TaskManager.Task::(prepare) Task=`a6085ac8-3dc7-4bea-9ce9-2286f9030dee`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000642887', 'lastCheck': '8.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:35:16,913::task::595::TaskManager.Task::(_updateState) Task=`a6085ac8-3dc7-4bea-9ce9-2286f9030dee`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:35:16,913::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:35:16,913::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:35:16,913::task::990::TaskManager.Task::(_decref) Task=`a6085ac8-3dc7-4bea-9ce9-2286f9030dee`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:35:17,285::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:17,364::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365625 s, 28.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:35:18,138::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:35:18,146::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000674201 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:35:19,377::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:19,457::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373412 s, 27.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:35:19,597::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:35:19,984::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:35:19,984::task::595::TaskManager.Task::(_updateState) Task=`7ecbdec3-73fa-42aa-977a-0b17a83a1169`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:35:19,984::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:35:19,985::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:35:19,985::task::1185::TaskManager.Task::(prepare) Task=`7ecbdec3-73fa-42aa-977a-0b17a83a1169`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:35:19,985::task::595::TaskManager.Task::(_updateState) Task=`7ecbdec3-73fa-42aa-977a-0b17a83a1169`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:35:19,985::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:35:19,985::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:35:19,986::task::990::TaskManager.Task::(_decref) Task=`7ecbdec3-73fa-42aa-977a-0b17a83a1169`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:35:19,990::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:35:19,990::task::595::TaskManager.Task::(_updateState) Task=`a50a8c7a-532c-48ac-9cd4-33c2d5a869c3`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:35:19,990::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:35:19,991::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1b17b128-02e3-4105-a7d2-5a3f85a8b64d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:35:19,991::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:35:19,991::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:35:19,992::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1b17b128-02e3-4105-a7d2-5a3f85a8b64d`::Granted request >Thread-338::DEBUG::2014-05-13 11:35:19,992::task::827::TaskManager.Task::(resourceAcquired) Task=`a50a8c7a-532c-48ac-9cd4-33c2d5a869c3`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:35:19,992::task::990::TaskManager.Task::(_decref) Task=`a50a8c7a-532c-48ac-9cd4-33c2d5a869c3`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:35:19,993::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:35:19,993::task::1185::TaskManager.Task::(prepare) Task=`a50a8c7a-532c-48ac-9cd4-33c2d5a869c3`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:35:19,993::task::595::TaskManager.Task::(_updateState) Task=`a50a8c7a-532c-48ac-9cd4-33c2d5a869c3`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:35:19,993::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:35:19,994::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:35:19,994::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:35:19,994::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:35:19,994::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:35:19,994::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:35:19,995::task::990::TaskManager.Task::(_decref) Task=`a50a8c7a-532c-48ac-9cd4-33c2d5a869c3`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:35:21,470::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:21,875::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.360351 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:35:22,616::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:35:23,888::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:23,963::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317285 s, 32.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:35:25,634::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:35:25,975::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:26,055::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358285 s, 28.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:35:28,068::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:28,141::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0299484 s, 34.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:35:28,169::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:35:28,177::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00060972 s, 6.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:35:28,651::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:35:30,018::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5eff0b16] >Thread-13::DEBUG::2014-05-13 11:35:30,019::task::595::TaskManager.Task::(_updateState) Task=`468ac774-3c09-4201-a223-90a061b900a8`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:35:30,019::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:35:30,019::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:35:30,019::task::1185::TaskManager.Task::(prepare) Task=`468ac774-3c09-4201-a223-90a061b900a8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:35:30,019::task::595::TaskManager.Task::(_updateState) Task=`468ac774-3c09-4201-a223-90a061b900a8`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:35:30,020::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:35:30,020::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:35:30,020::task::990::TaskManager.Task::(_decref) Task=`468ac774-3c09-4201-a223-90a061b900a8`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:35:30,024::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5eff0b16] >Thread-338::DEBUG::2014-05-13 11:35:30,024::task::595::TaskManager.Task::(_updateState) Task=`e5986305-2802-4565-9594-61300ccdbd55`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:35:30,024::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:35:30,025::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3e7fcabc-cd0a-4ff7-b403-5c3434ea7c49`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:35:30,025::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:35:30,025::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:35:30,025::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3e7fcabc-cd0a-4ff7-b403-5c3434ea7c49`::Granted request >Thread-338::DEBUG::2014-05-13 11:35:30,026::task::827::TaskManager.Task::(resourceAcquired) Task=`e5986305-2802-4565-9594-61300ccdbd55`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:35:30,026::task::990::TaskManager.Task::(_decref) Task=`e5986305-2802-4565-9594-61300ccdbd55`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:35:30,026::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:35:30,027::task::1185::TaskManager.Task::(prepare) Task=`e5986305-2802-4565-9594-61300ccdbd55`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:35:30,027::task::595::TaskManager.Task::(_updateState) Task=`e5986305-2802-4565-9594-61300ccdbd55`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:35:30,027::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:35:30,027::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:35:30,027::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:35:30,027::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:35:30,028::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:35:30,028::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:35:30,028::task::990::TaskManager.Task::(_decref) Task=`e5986305-2802-4565-9594-61300ccdbd55`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:35:30,178::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:30,253::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307115 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:35:31,682::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:35:32,095::task::595::TaskManager.Task::(_updateState) Task=`387b419f-2074-44ce-9af2-a154f904ccff`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:35:32,095::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:35:32,095::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00060972', 'lastCheck': '3.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:35:32,096::task::1185::TaskManager.Task::(prepare) Task=`387b419f-2074-44ce-9af2-a154f904ccff`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00060972', 'lastCheck': '3.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:35:32,096::task::595::TaskManager.Task::(_updateState) Task=`387b419f-2074-44ce-9af2-a154f904ccff`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:35:32,096::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:35:32,096::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:35:32,096::task::990::TaskManager.Task::(_decref) Task=`387b419f-2074-44ce-9af2-a154f904ccff`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:35:32,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:32,357::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0484386 s, 21.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:35:34,370::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:34,449::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356511 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:35:34,702::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:35:36,462::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:36,535::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0301466 s, 34.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:35:37,721::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:35:38,185::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:35:38,197::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000622116 s, 6.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:35:38,548::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:38,623::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0335328 s, 30.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:35:40,049::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:35:40,049::task::595::TaskManager.Task::(_updateState) Task=`43fc6a0f-449e-4863-94c1-914dd1035cf8`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:35:40,050::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:35:40,050::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:35:40,050::task::1185::TaskManager.Task::(prepare) Task=`43fc6a0f-449e-4863-94c1-914dd1035cf8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:35:40,050::task::595::TaskManager.Task::(_updateState) Task=`43fc6a0f-449e-4863-94c1-914dd1035cf8`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:35:40,050::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:35:40,051::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:35:40,051::task::990::TaskManager.Task::(_decref) Task=`43fc6a0f-449e-4863-94c1-914dd1035cf8`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:35:40,055::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:35:40,055::task::595::TaskManager.Task::(_updateState) Task=`89015df8-60ad-4e38-93a4-9084e81da6ef`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:35:40,055::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:35:40,056::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`12f50152-18c7-42b9-8524-081e243d2434`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:35:40,056::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:35:40,056::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:35:40,057::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`12f50152-18c7-42b9-8524-081e243d2434`::Granted request >Thread-338::DEBUG::2014-05-13 11:35:40,057::task::827::TaskManager.Task::(resourceAcquired) Task=`89015df8-60ad-4e38-93a4-9084e81da6ef`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:35:40,057::task::990::TaskManager.Task::(_decref) Task=`89015df8-60ad-4e38-93a4-9084e81da6ef`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:35:40,058::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:35:40,058::task::1185::TaskManager.Task::(prepare) Task=`89015df8-60ad-4e38-93a4-9084e81da6ef`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:35:40,058::task::595::TaskManager.Task::(_updateState) Task=`89015df8-60ad-4e38-93a4-9084e81da6ef`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:35:40,058::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:35:40,059::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:35:40,059::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:35:40,059::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:35:40,059::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:35:40,059::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:35:40,060::task::990::TaskManager.Task::(_decref) Task=`89015df8-60ad-4e38-93a4-9084e81da6ef`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:35:40,636::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:40,716::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361186 s, 28.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:35:40,741::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:35:42,729::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:42,806::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0334762 s, 30.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:35:43,760::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:35:44,819::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:44,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319731 s, 32.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:35:46,779::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:35:46,907::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 11:35:47,234::task::595::TaskManager.Task::(_updateState) Task=`4d3deea5-6b33-4b82-aa0f-6db1a886b723`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:35:47,234::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:35:47,234::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000622116', 'lastCheck': '9.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:35:47,234::task::1185::TaskManager.Task::(prepare) Task=`4d3deea5-6b33-4b82-aa0f-6db1a886b723`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000622116', 'lastCheck': '9.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:35:47,235::task::595::TaskManager.Task::(_updateState) Task=`4d3deea5-6b33-4b82-aa0f-6db1a886b723`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:35:47,235::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:35:47,235::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:35:47,235::task::990::TaskManager.Task::(_decref) Task=`4d3deea5-6b33-4b82-aa0f-6db1a886b723`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:35:47,360::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.410524 s, 2.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:35:48,204::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:35:48,213::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00144042 s, 2.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:35:49,373::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:49,452::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357214 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:35:49,797::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:35:50,076::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [19876d15] >Thread-13::DEBUG::2014-05-13 11:35:50,077::task::595::TaskManager.Task::(_updateState) Task=`a92a5855-5e1b-49d4-977f-ef83c96301e5`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:35:50,077::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:35:50,077::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:35:50,077::task::1185::TaskManager.Task::(prepare) Task=`a92a5855-5e1b-49d4-977f-ef83c96301e5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:35:50,078::task::595::TaskManager.Task::(_updateState) Task=`a92a5855-5e1b-49d4-977f-ef83c96301e5`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:35:50,078::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:35:50,078::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:35:50,078::task::990::TaskManager.Task::(_decref) Task=`a92a5855-5e1b-49d4-977f-ef83c96301e5`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:35:50,083::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [19876d15] >Thread-338::DEBUG::2014-05-13 11:35:50,083::task::595::TaskManager.Task::(_updateState) Task=`55a84135-28e4-4584-b107-7424cc71f6da`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:35:50,084::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:35:50,084::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`afce4cc0-6586-4561-9dab-e259511df61a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:35:50,084::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:35:50,084::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:35:50,085::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`afce4cc0-6586-4561-9dab-e259511df61a`::Granted request >Thread-338::DEBUG::2014-05-13 11:35:50,085::task::827::TaskManager.Task::(resourceAcquired) Task=`55a84135-28e4-4584-b107-7424cc71f6da`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:35:50,085::task::990::TaskManager.Task::(_decref) Task=`55a84135-28e4-4584-b107-7424cc71f6da`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:35:50,086::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:35:50,086::task::1185::TaskManager.Task::(prepare) Task=`55a84135-28e4-4584-b107-7424cc71f6da`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:35:50,086::task::595::TaskManager.Task::(_updateState) Task=`55a84135-28e4-4584-b107-7424cc71f6da`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:35:50,086::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:35:50,087::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:35:50,087::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:35:50,087::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:35:50,087::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:35:50,087::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:35:50,088::task::990::TaskManager.Task::(_decref) Task=`55a84135-28e4-4584-b107-7424cc71f6da`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:35:51,465::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:51,540::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303046 s, 33.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:35:52,814::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:35:53,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:53,627::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306601 s, 33.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:35:55,640::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:55,720::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356397 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:35:55,836::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:35:57,732::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:35:57,806::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305451 s, 33.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:35:58,220::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:35:58,229::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000671656 s, 6.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:35:58,854::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:35:59,819::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 11:36:00,105::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:36:00,106::task::595::TaskManager.Task::(_updateState) Task=`46048547-4451-4481-9e2c-041468ddcc9f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:36:00,106::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:36:00,106::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:36:00,106::task::1185::TaskManager.Task::(prepare) Task=`46048547-4451-4481-9e2c-041468ddcc9f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:36:00,107::task::595::TaskManager.Task::(_updateState) Task=`46048547-4451-4481-9e2c-041468ddcc9f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:36:00,107::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:36:00,107::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:36:00,107::task::990::TaskManager.Task::(_decref) Task=`46048547-4451-4481-9e2c-041468ddcc9f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:36:00,112::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:36:00,112::task::595::TaskManager.Task::(_updateState) Task=`d697b1db-b137-467d-9b7d-34f39cc9abfa`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:36:00,112::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:36:00,113::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9704e99c-a08b-496b-a4cb-8b6effd9bf1e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:36:00,113::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:36:00,113::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:36:00,113::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9704e99c-a08b-496b-a4cb-8b6effd9bf1e`::Granted request >Thread-338::DEBUG::2014-05-13 11:36:00,113::task::827::TaskManager.Task::(resourceAcquired) Task=`d697b1db-b137-467d-9b7d-34f39cc9abfa`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:36:00,114::task::990::TaskManager.Task::(_decref) Task=`d697b1db-b137-467d-9b7d-34f39cc9abfa`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:36:00,114::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:36:00,114::task::1185::TaskManager.Task::(prepare) Task=`d697b1db-b137-467d-9b7d-34f39cc9abfa`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:36:00,115::task::595::TaskManager.Task::(_updateState) Task=`d697b1db-b137-467d-9b7d-34f39cc9abfa`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:36:00,115::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:36:00,115::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:36:00,115::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:36:00,115::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:36:00,115::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:36:00,115::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:36:00,116::task::990::TaskManager.Task::(_decref) Task=`d697b1db-b137-467d-9b7d-34f39cc9abfa`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:36:00,180::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.317015 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:36:01,871::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:36:02,193::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:02,272::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0348576 s, 29.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:36:02,370::task::595::TaskManager.Task::(_updateState) Task=`35ffae59-a96d-44ba-a46b-6d11ff49f1e7`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:36:02,370::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:36:02,370::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000671656', 'lastCheck': '4.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:36:02,371::task::1185::TaskManager.Task::(prepare) Task=`35ffae59-a96d-44ba-a46b-6d11ff49f1e7`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000671656', 'lastCheck': '4.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:36:02,371::task::595::TaskManager.Task::(_updateState) Task=`35ffae59-a96d-44ba-a46b-6d11ff49f1e7`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:36:02,371::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:36:02,371::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:36:02,371::task::990::TaskManager.Task::(_decref) Task=`35ffae59-a96d-44ba-a46b-6d11ff49f1e7`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:36:04,285::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:04,359::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311887 s, 32.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:36:04,889::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:36:06,376::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:06,452::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320748 s, 31.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:36:07,907::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:36:08,236::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:36:08,244::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000623768 s, 6.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:36:08,465::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:08,544::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353078 s, 29.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:36:10,134::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:36:10,136::task::595::TaskManager.Task::(_updateState) Task=`7cd0b709-b094-4bb0-bfaf-c25f17abaa7c`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:36:10,136::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:36:10,136::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:36:10,137::task::1185::TaskManager.Task::(prepare) Task=`7cd0b709-b094-4bb0-bfaf-c25f17abaa7c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:36:10,137::task::595::TaskManager.Task::(_updateState) Task=`7cd0b709-b094-4bb0-bfaf-c25f17abaa7c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:36:10,137::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:36:10,137::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:36:10,137::task::990::TaskManager.Task::(_decref) Task=`7cd0b709-b094-4bb0-bfaf-c25f17abaa7c`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:36:10,142::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:36:10,142::task::595::TaskManager.Task::(_updateState) Task=`5ecc7f95-44a4-46b5-94e1-153c91bedb2c`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:36:10,143::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:36:10,143::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`14c7a002-e920-4cba-b906-249988d0975a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:36:10,143::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:36:10,144::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:36:10,144::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`14c7a002-e920-4cba-b906-249988d0975a`::Granted request >Thread-338::DEBUG::2014-05-13 11:36:10,144::task::827::TaskManager.Task::(resourceAcquired) Task=`5ecc7f95-44a4-46b5-94e1-153c91bedb2c`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:36:10,144::task::990::TaskManager.Task::(_decref) Task=`5ecc7f95-44a4-46b5-94e1-153c91bedb2c`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:36:10,145::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:36:10,145::task::1185::TaskManager.Task::(prepare) Task=`5ecc7f95-44a4-46b5-94e1-153c91bedb2c`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:36:10,145::task::595::TaskManager.Task::(_updateState) Task=`5ecc7f95-44a4-46b5-94e1-153c91bedb2c`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:36:10,145::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:36:10,146::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:36:10,146::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:36:10,146::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:36:10,146::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:36:10,146::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:36:10,147::task::990::TaskManager.Task::(_decref) Task=`5ecc7f95-44a4-46b5-94e1-153c91bedb2c`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:36:10,557::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:36:10,928::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:36:10,972::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.371196 s, 2.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:36:12,985::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:13,064::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364399 s, 28.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:36:13,946::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:36:15,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:15,152::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309607 s, 33.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:36:16,967::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:36:17,165::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:17,243::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0347849 s, 29.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:36:17,517::task::595::TaskManager.Task::(_updateState) Task=`43bcdcb7-b721-4c15-9d09-85d31ccb02cd`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:36:17,517::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:36:17,518::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000623768', 'lastCheck': '9.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:36:17,518::task::1185::TaskManager.Task::(prepare) Task=`43bcdcb7-b721-4c15-9d09-85d31ccb02cd`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000623768', 'lastCheck': '9.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:36:17,518::task::595::TaskManager.Task::(_updateState) Task=`43bcdcb7-b721-4c15-9d09-85d31ccb02cd`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:36:17,518::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:36:17,518::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:36:17,519::task::990::TaskManager.Task::(_decref) Task=`43bcdcb7-b721-4c15-9d09-85d31ccb02cd`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 11:36:18,251::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:36:18,260::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000695331 s, 5.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:36:19,256::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:19,334::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035196 s, 29.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:36:19,986::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:36:20,164::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:36:20,165::task::595::TaskManager.Task::(_updateState) Task=`72eba61b-b434-4d0a-b465-6df378b27abf`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:36:20,165::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:36:20,165::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:36:20,166::task::1185::TaskManager.Task::(prepare) Task=`72eba61b-b434-4d0a-b465-6df378b27abf`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:36:20,166::task::595::TaskManager.Task::(_updateState) Task=`72eba61b-b434-4d0a-b465-6df378b27abf`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:36:20,166::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:36:20,166::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:36:20,166::task::990::TaskManager.Task::(_decref) Task=`72eba61b-b434-4d0a-b465-6df378b27abf`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:36:20,172::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:36:20,173::task::595::TaskManager.Task::(_updateState) Task=`531faf97-71c9-42bb-9b44-e17eff9d6a52`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:36:20,173::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:36:20,173::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a04f6f5c-d052-4d4b-860b-76a4086f1687`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:36:20,173::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:36:20,174::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:36:20,174::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a04f6f5c-d052-4d4b-860b-76a4086f1687`::Granted request >Thread-338::DEBUG::2014-05-13 11:36:20,174::task::827::TaskManager.Task::(resourceAcquired) Task=`531faf97-71c9-42bb-9b44-e17eff9d6a52`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:36:20,174::task::990::TaskManager.Task::(_decref) Task=`531faf97-71c9-42bb-9b44-e17eff9d6a52`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:36:20,175::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:36:20,175::task::1185::TaskManager.Task::(prepare) Task=`531faf97-71c9-42bb-9b44-e17eff9d6a52`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:36:20,176::task::595::TaskManager.Task::(_updateState) Task=`531faf97-71c9-42bb-9b44-e17eff9d6a52`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:36:20,176::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:36:20,176::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:36:20,176::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:36:20,176::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:36:20,177::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:36:20,177::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:36:20,177::task::990::TaskManager.Task::(_decref) Task=`531faf97-71c9-42bb-9b44-e17eff9d6a52`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:36:21,347::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:21,424::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031488 s, 32.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:36:23,009::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:36:23,437::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:23,516::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358358 s, 28.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:36:25,530::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:25,609::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358641 s, 28.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:36:26,027::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:36:27,622::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:27,975::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.310082 s, 3.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:36:28,267::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:36:28,276::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000756006 s, 5.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:36:29,049::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:36:29,988::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:30,064::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320271 s, 32.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:36:30,198::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:36:30,198::task::595::TaskManager.Task::(_updateState) Task=`41d41c47-282e-4cf2-b15c-e08553dfa39b`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:36:30,198::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:36:30,199::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:36:30,199::task::1185::TaskManager.Task::(prepare) Task=`41d41c47-282e-4cf2-b15c-e08553dfa39b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:36:30,199::task::595::TaskManager.Task::(_updateState) Task=`41d41c47-282e-4cf2-b15c-e08553dfa39b`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:36:30,199::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:36:30,199::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:36:30,200::task::990::TaskManager.Task::(_decref) Task=`41d41c47-282e-4cf2-b15c-e08553dfa39b`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:36:30,204::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:36:30,204::task::595::TaskManager.Task::(_updateState) Task=`66b6ba5d-36f9-4d3f-a591-d1a1e22c3a9a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:36:30,204::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:36:30,205::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9f1cc20e-6dbc-443e-9a74-660a515a22d9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:36:30,205::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:36:30,205::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:36:30,206::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9f1cc20e-6dbc-443e-9a74-660a515a22d9`::Granted request >Thread-338::DEBUG::2014-05-13 11:36:30,206::task::827::TaskManager.Task::(resourceAcquired) Task=`66b6ba5d-36f9-4d3f-a591-d1a1e22c3a9a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:36:30,206::task::990::TaskManager.Task::(_decref) Task=`66b6ba5d-36f9-4d3f-a591-d1a1e22c3a9a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:36:30,207::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:36:30,207::task::1185::TaskManager.Task::(prepare) Task=`66b6ba5d-36f9-4d3f-a591-d1a1e22c3a9a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:36:30,207::task::595::TaskManager.Task::(_updateState) Task=`66b6ba5d-36f9-4d3f-a591-d1a1e22c3a9a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:36:30,207::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:36:30,208::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:36:30,208::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:36:30,208::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:36:30,208::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:36:30,208::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:36:30,209::task::990::TaskManager.Task::(_decref) Task=`66b6ba5d-36f9-4d3f-a591-d1a1e22c3a9a`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:36:32,069::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:36:32,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:32,157::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364193 s, 28.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:36:32,648::task::595::TaskManager.Task::(_updateState) Task=`67807165-a777-45c0-a030-4c0d36f1efc2`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:36:32,648::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:36:32,649::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000756006', 'lastCheck': '4.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:36:32,649::task::1185::TaskManager.Task::(prepare) Task=`67807165-a777-45c0-a030-4c0d36f1efc2`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000756006', 'lastCheck': '4.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:36:32,649::task::595::TaskManager.Task::(_updateState) Task=`67807165-a777-45c0-a030-4c0d36f1efc2`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:36:32,649::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:36:32,649::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:36:32,649::task::990::TaskManager.Task::(_decref) Task=`67807165-a777-45c0-a030-4c0d36f1efc2`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:36:34,170::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:34,247::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0337869 s, 30.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:36:35,092::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:36:36,260::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:36,336::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315609 s, 32.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:36:38,110::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:36:38,307::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:36:38,316::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00065191 s, 6.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:36:38,349::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:38,442::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0489011 s, 20.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:36:40,227::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:36:40,228::task::595::TaskManager.Task::(_updateState) Task=`b7680b4b-ec74-46d1-95f7-67f0cd00752f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:36:40,228::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:36:40,228::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:36:40,228::task::1185::TaskManager.Task::(prepare) Task=`b7680b4b-ec74-46d1-95f7-67f0cd00752f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:36:40,228::task::595::TaskManager.Task::(_updateState) Task=`b7680b4b-ec74-46d1-95f7-67f0cd00752f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:36:40,229::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:36:40,229::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:36:40,229::task::990::TaskManager.Task::(_decref) Task=`b7680b4b-ec74-46d1-95f7-67f0cd00752f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:36:40,234::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:36:40,234::task::595::TaskManager.Task::(_updateState) Task=`21fa0e2b-9f57-45e3-a979-db6e45f1f683`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:36:40,234::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:36:40,235::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5fe847e3-438d-4ceb-a08a-1913c7ab87f2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:36:40,235::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:36:40,235::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:36:40,235::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5fe847e3-438d-4ceb-a08a-1913c7ab87f2`::Granted request >Thread-338::DEBUG::2014-05-13 11:36:40,236::task::827::TaskManager.Task::(resourceAcquired) Task=`21fa0e2b-9f57-45e3-a979-db6e45f1f683`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:36:40,236::task::990::TaskManager.Task::(_decref) Task=`21fa0e2b-9f57-45e3-a979-db6e45f1f683`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:36:40,237::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:36:40,237::task::1185::TaskManager.Task::(prepare) Task=`21fa0e2b-9f57-45e3-a979-db6e45f1f683`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:36:40,237::task::595::TaskManager.Task::(_updateState) Task=`21fa0e2b-9f57-45e3-a979-db6e45f1f683`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:36:40,237::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:36:40,237::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:36:40,238::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:36:40,238::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:36:40,238::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:36:40,238::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:36:40,238::task::990::TaskManager.Task::(_decref) Task=`21fa0e2b-9f57-45e3-a979-db6e45f1f683`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:36:40,455::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:40,534::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352242 s, 29.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:36:41,132::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:36:42,547::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:42,623::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317099 s, 32.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:36:44,153::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:36:44,636::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:44,712::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309137 s, 33.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:36:46,725::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:46,804::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353486 s, 29.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:36:47,175::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:36:47,779::task::595::TaskManager.Task::(_updateState) Task=`0d9e4caa-1c49-48e7-92ac-c746f24bd241`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:36:47,779::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:36:47,779::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00065191', 'lastCheck': '9.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:36:47,780::task::1185::TaskManager.Task::(prepare) Task=`0d9e4caa-1c49-48e7-92ac-c746f24bd241`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00065191', 'lastCheck': '9.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:36:47,780::task::595::TaskManager.Task::(_updateState) Task=`0d9e4caa-1c49-48e7-92ac-c746f24bd241`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:36:47,780::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:36:47,780::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:36:47,780::task::990::TaskManager.Task::(_decref) Task=`0d9e4caa-1c49-48e7-92ac-c746f24bd241`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 11:36:48,322::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:36:48,331::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000619853 s, 6.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:36:48,817::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:48,896::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0351052 s, 29.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:36:50,195::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:36:50,261::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:36:50,262::task::595::TaskManager.Task::(_updateState) Task=`b3a2ee12-1597-4a7d-b897-8a3c4d9f03a0`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:36:50,262::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:36:50,262::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:36:50,263::task::1185::TaskManager.Task::(prepare) Task=`b3a2ee12-1597-4a7d-b897-8a3c4d9f03a0`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:36:50,263::task::595::TaskManager.Task::(_updateState) Task=`b3a2ee12-1597-4a7d-b897-8a3c4d9f03a0`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:36:50,263::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:36:50,263::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:36:50,263::task::990::TaskManager.Task::(_decref) Task=`b3a2ee12-1597-4a7d-b897-8a3c4d9f03a0`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:36:50,268::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:36:50,268::task::595::TaskManager.Task::(_updateState) Task=`0edf7345-92d0-472a-a983-e222e8010c23`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:36:50,268::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:36:50,269::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9799a5b3-c20f-42c4-b9ea-1e17f5979410`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:36:50,269::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:36:50,269::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:36:50,270::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9799a5b3-c20f-42c4-b9ea-1e17f5979410`::Granted request >Thread-338::DEBUG::2014-05-13 11:36:50,270::task::827::TaskManager.Task::(resourceAcquired) Task=`0edf7345-92d0-472a-a983-e222e8010c23`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:36:50,270::task::990::TaskManager.Task::(_decref) Task=`0edf7345-92d0-472a-a983-e222e8010c23`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:36:50,271::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:36:50,271::task::1185::TaskManager.Task::(prepare) Task=`0edf7345-92d0-472a-a983-e222e8010c23`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:36:50,271::task::595::TaskManager.Task::(_updateState) Task=`0edf7345-92d0-472a-a983-e222e8010c23`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:36:50,271::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:36:50,272::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:36:50,272::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:36:50,272::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:36:50,272::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:36:50,272::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:36:50,273::task::990::TaskManager.Task::(_decref) Task=`0edf7345-92d0-472a-a983-e222e8010c23`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:36:50,909::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:50,984::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0328292 s, 31.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:36:52,997::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:53,088::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0479294 s, 21.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:36:53,214::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:36:55,101::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:55,183::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0375084 s, 27.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:36:56,233::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:36:57,195::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:57,269::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0301172 s, 34.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:36:58,338::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:36:58,347::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000664977 s, 6.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:36:59,254::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:36:59,282::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:36:59,362::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0368985 s, 27.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:37:00,292::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3e537ec9] >Thread-13::DEBUG::2014-05-13 11:37:00,292::task::595::TaskManager.Task::(_updateState) Task=`39306066-7d2b-488d-8d55-77994c6379c1`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:37:00,293::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:37:00,293::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:37:00,293::task::1185::TaskManager.Task::(prepare) Task=`39306066-7d2b-488d-8d55-77994c6379c1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:37:00,293::task::595::TaskManager.Task::(_updateState) Task=`39306066-7d2b-488d-8d55-77994c6379c1`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:37:00,293::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:37:00,294::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:37:00,294::task::990::TaskManager.Task::(_decref) Task=`39306066-7d2b-488d-8d55-77994c6379c1`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:37:00,299::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3e537ec9] >Thread-338::DEBUG::2014-05-13 11:37:00,299::task::595::TaskManager.Task::(_updateState) Task=`e91de929-0769-4b60-ac48-fb4eea1c30cb`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:37:00,299::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:37:00,300::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3a2124cb-2fdf-47ee-8103-467c9ce5892a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:37:00,300::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:37:00,300::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:37:00,300::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3a2124cb-2fdf-47ee-8103-467c9ce5892a`::Granted request >Thread-338::DEBUG::2014-05-13 11:37:00,300::task::827::TaskManager.Task::(resourceAcquired) Task=`e91de929-0769-4b60-ac48-fb4eea1c30cb`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:37:00,301::task::990::TaskManager.Task::(_decref) Task=`e91de929-0769-4b60-ac48-fb4eea1c30cb`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:37:00,301::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:37:00,302::task::1185::TaskManager.Task::(prepare) Task=`e91de929-0769-4b60-ac48-fb4eea1c30cb`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:37:00,302::task::595::TaskManager.Task::(_updateState) Task=`e91de929-0769-4b60-ac48-fb4eea1c30cb`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:37:00,302::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:37:00,302::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:37:00,302::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:37:00,303::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:37:00,303::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:37:00,303::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:37:00,303::task::990::TaskManager.Task::(_decref) Task=`e91de929-0769-4b60-ac48-fb4eea1c30cb`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:37:01,375::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:01,454::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035549 s, 28.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:37:02,272::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:37:02,909::task::595::TaskManager.Task::(_updateState) Task=`3d767b7f-2f99-44f8-9771-bb867b23c19c`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:37:02,909::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:37:02,909::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000664977', 'lastCheck': '4.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:37:02,909::task::1185::TaskManager.Task::(prepare) Task=`3d767b7f-2f99-44f8-9771-bb867b23c19c`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000664977', 'lastCheck': '4.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:37:02,909::task::595::TaskManager.Task::(_updateState) Task=`3d767b7f-2f99-44f8-9771-bb867b23c19c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:37:02,910::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:37:02,910::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:37:02,910::task::990::TaskManager.Task::(_decref) Task=`3d767b7f-2f99-44f8-9771-bb867b23c19c`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:37:03,467::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:03,541::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316375 s, 32.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:37:05,290::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:37:05,554::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:05,630::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358622 s, 28.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:37:07,638::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:08,038::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.357147 s, 2.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:37:08,309::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:37:08,371::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:37:08,380::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000753312 s, 5.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:37:10,051::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:10,144::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0496641 s, 20.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:37:10,323::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:37:10,324::task::595::TaskManager.Task::(_updateState) Task=`93db48bb-bff5-48ea-90dc-515095d0ebc1`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:37:10,324::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:37:10,324::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:37:10,324::task::1185::TaskManager.Task::(prepare) Task=`93db48bb-bff5-48ea-90dc-515095d0ebc1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:37:10,325::task::595::TaskManager.Task::(_updateState) Task=`93db48bb-bff5-48ea-90dc-515095d0ebc1`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:37:10,325::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:37:10,325::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:37:10,325::task::990::TaskManager.Task::(_decref) Task=`93db48bb-bff5-48ea-90dc-515095d0ebc1`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:37:10,329::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:37:10,329::task::595::TaskManager.Task::(_updateState) Task=`a5e38d27-c933-42e7-af3e-816854e5804b`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:37:10,330::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:37:10,330::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6abc1e24-8531-43cb-845e-ee4285d828d0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:37:10,330::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:37:10,331::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:37:10,331::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6abc1e24-8531-43cb-845e-ee4285d828d0`::Granted request >Thread-338::DEBUG::2014-05-13 11:37:10,331::task::827::TaskManager.Task::(resourceAcquired) Task=`a5e38d27-c933-42e7-af3e-816854e5804b`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:37:10,331::task::990::TaskManager.Task::(_decref) Task=`a5e38d27-c933-42e7-af3e-816854e5804b`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:37:10,332::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:37:10,332::task::1185::TaskManager.Task::(prepare) Task=`a5e38d27-c933-42e7-af3e-816854e5804b`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:37:10,332::task::595::TaskManager.Task::(_updateState) Task=`a5e38d27-c933-42e7-af3e-816854e5804b`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:37:10,333::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:37:10,333::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:37:10,333::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:37:10,333::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:37:10,333::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:37:10,333::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:37:10,334::task::990::TaskManager.Task::(_decref) Task=`a5e38d27-c933-42e7-af3e-816854e5804b`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:37:11,328::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:37:12,157::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:12,233::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318434 s, 32.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:37:14,246::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:14,328::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0380684 s, 26.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:37:14,347::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:37:16,341::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:16,421::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357444 s, 28.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:37:17,367::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:37:18,046::task::595::TaskManager.Task::(_updateState) Task=`1a32df25-567d-45ce-b822-c8d5430232df`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:37:18,046::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:37:18,046::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000753312', 'lastCheck': '9.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:37:18,046::task::1185::TaskManager.Task::(prepare) Task=`1a32df25-567d-45ce-b822-c8d5430232df`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000753312', 'lastCheck': '9.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:37:18,046::task::595::TaskManager.Task::(_updateState) Task=`1a32df25-567d-45ce-b822-c8d5430232df`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:37:18,047::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:37:18,047::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:37:18,047::task::990::TaskManager.Task::(_decref) Task=`1a32df25-567d-45ce-b822-c8d5430232df`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 11:37:18,387::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:37:18,395::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000695692 s, 5.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:37:18,434::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:18,844::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.367416 s, 2.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:37:20,356::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:37:20,357::task::595::TaskManager.Task::(_updateState) Task=`a2ee7946-3703-4000-9992-e8589938cf69`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:37:20,357::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:37:20,357::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:37:20,357::task::1185::TaskManager.Task::(prepare) Task=`a2ee7946-3703-4000-9992-e8589938cf69`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:37:20,358::task::595::TaskManager.Task::(_updateState) Task=`a2ee7946-3703-4000-9992-e8589938cf69`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:37:20,358::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:37:20,358::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:37:20,358::task::990::TaskManager.Task::(_decref) Task=`a2ee7946-3703-4000-9992-e8589938cf69`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:37:20,363::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:37:20,364::task::595::TaskManager.Task::(_updateState) Task=`13150fb1-7ce5-40bb-ba3a-6e40314c4782`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:37:20,364::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:37:20,364::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`92dd828b-8914-4ef2-b31d-508044b469c9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:37:20,364::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:37:20,365::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:37:20,365::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`92dd828b-8914-4ef2-b31d-508044b469c9`::Granted request >Thread-338::DEBUG::2014-05-13 11:37:20,365::task::827::TaskManager.Task::(resourceAcquired) Task=`13150fb1-7ce5-40bb-ba3a-6e40314c4782`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:37:20,365::task::990::TaskManager.Task::(_decref) Task=`13150fb1-7ce5-40bb-ba3a-6e40314c4782`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:37:20,366::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:37:20,366::task::1185::TaskManager.Task::(prepare) Task=`13150fb1-7ce5-40bb-ba3a-6e40314c4782`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:37:20,366::task::595::TaskManager.Task::(_updateState) Task=`13150fb1-7ce5-40bb-ba3a-6e40314c4782`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:37:20,367::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:37:20,367::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:37:20,367::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:37:20,367::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:37:20,367::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:37:20,368::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:37:20,368::task::990::TaskManager.Task::(_decref) Task=`13150fb1-7ce5-40bb-ba3a-6e40314c4782`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:37:20,385::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:37:20,857::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:20,931::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0300977 s, 34.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:37:22,944::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:23,023::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0350883 s, 29.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:37:23,419::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:37:25,036::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:25,117::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0371749 s, 27.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:37:26,436::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:37:27,130::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:27,205::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323513 s, 31.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:37:28,403::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:37:28,411::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000799802 s, 5.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:37:29,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:29,299::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374342 s, 27.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:37:29,454::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:37:30,384::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:37:30,385::task::595::TaskManager.Task::(_updateState) Task=`518890e0-f527-48a5-be9b-4869dcabafa0`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:37:30,385::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:37:30,385::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:37:30,385::task::1185::TaskManager.Task::(prepare) Task=`518890e0-f527-48a5-be9b-4869dcabafa0`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:37:30,386::task::595::TaskManager.Task::(_updateState) Task=`518890e0-f527-48a5-be9b-4869dcabafa0`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:37:30,386::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:37:30,386::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:37:30,386::task::990::TaskManager.Task::(_decref) Task=`518890e0-f527-48a5-be9b-4869dcabafa0`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:37:30,391::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:37:30,391::task::595::TaskManager.Task::(_updateState) Task=`b6fe5094-f2f8-4136-9d7b-6d95a9d52fae`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:37:30,391::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:37:30,391::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fca0e973-ec62-409b-ae9a-f3582cf5144c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:37:30,392::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:37:30,392::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:37:30,392::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fca0e973-ec62-409b-ae9a-f3582cf5144c`::Granted request >Thread-338::DEBUG::2014-05-13 11:37:30,392::task::827::TaskManager.Task::(resourceAcquired) Task=`b6fe5094-f2f8-4136-9d7b-6d95a9d52fae`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:37:30,393::task::990::TaskManager.Task::(_decref) Task=`b6fe5094-f2f8-4136-9d7b-6d95a9d52fae`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:37:30,393::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:37:30,394::task::1185::TaskManager.Task::(prepare) Task=`b6fe5094-f2f8-4136-9d7b-6d95a9d52fae`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:37:30,394::task::595::TaskManager.Task::(_updateState) Task=`b6fe5094-f2f8-4136-9d7b-6d95a9d52fae`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:37:30,394::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:37:30,394::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:37:30,394::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:37:30,395::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:37:30,395::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:37:30,395::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:37:30,395::task::990::TaskManager.Task::(_decref) Task=`b6fe5094-f2f8-4136-9d7b-6d95a9d52fae`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:37:31,312::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:31,391::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0366309 s, 28.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:37:32,473::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:37:33,180::task::595::TaskManager.Task::(_updateState) Task=`03759a92-171b-4bf0-ad6e-84ad60686ae8`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:37:33,181::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:37:33,181::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000799802', 'lastCheck': '4.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:37:33,181::task::1185::TaskManager.Task::(prepare) Task=`03759a92-171b-4bf0-ad6e-84ad60686ae8`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000799802', 'lastCheck': '4.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:37:33,181::task::595::TaskManager.Task::(_updateState) Task=`03759a92-171b-4bf0-ad6e-84ad60686ae8`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:37:33,181::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:37:33,181::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:37:33,182::task::990::TaskManager.Task::(_decref) Task=`03759a92-171b-4bf0-ad6e-84ad60686ae8`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:37:33,404::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:33,494::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0456614 s, 22.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:37:35,492::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:37:35,507::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:35,585::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355887 s, 28.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:37:37,598::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:37,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317143 s, 32.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:37:38,418::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:37:38,427::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000649985 s, 6.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:37:38,511::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:37:39,686::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:39,762::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0333806 s, 30.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:37:40,414::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6019f20] >Thread-13::DEBUG::2014-05-13 11:37:40,415::task::595::TaskManager.Task::(_updateState) Task=`59e572cb-2bb3-4a3a-90dc-0eda7016a6d7`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:37:40,415::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:37:40,415::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:37:40,416::task::1185::TaskManager.Task::(prepare) Task=`59e572cb-2bb3-4a3a-90dc-0eda7016a6d7`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:37:40,416::task::595::TaskManager.Task::(_updateState) Task=`59e572cb-2bb3-4a3a-90dc-0eda7016a6d7`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:37:40,416::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:37:40,416::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:37:40,416::task::990::TaskManager.Task::(_decref) Task=`59e572cb-2bb3-4a3a-90dc-0eda7016a6d7`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:37:40,420::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6019f20] >Thread-338::DEBUG::2014-05-13 11:37:40,421::task::595::TaskManager.Task::(_updateState) Task=`348b8639-f73c-4465-9372-d03f6405fccb`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:37:40,421::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:37:40,421::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0df70758-f6a1-4ab2-b060-e8d9ead39c79`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:37:40,422::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:37:40,422::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:37:40,422::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0df70758-f6a1-4ab2-b060-e8d9ead39c79`::Granted request >Thread-338::DEBUG::2014-05-13 11:37:40,422::task::827::TaskManager.Task::(resourceAcquired) Task=`348b8639-f73c-4465-9372-d03f6405fccb`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:37:40,423::task::990::TaskManager.Task::(_decref) Task=`348b8639-f73c-4465-9372-d03f6405fccb`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:37:40,423::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:37:40,423::task::1185::TaskManager.Task::(prepare) Task=`348b8639-f73c-4465-9372-d03f6405fccb`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:37:40,424::task::595::TaskManager.Task::(_updateState) Task=`348b8639-f73c-4465-9372-d03f6405fccb`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:37:40,424::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:37:40,424::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:37:40,424::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:37:40,424::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:37:40,425::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:37:40,425::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:37:40,425::task::990::TaskManager.Task::(_decref) Task=`348b8639-f73c-4465-9372-d03f6405fccb`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:37:41,531::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:37:41,775::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:41,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0326846 s, 31.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:37:43,864::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:43,944::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374744 s, 27.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:37:44,551::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:37:45,957::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:46,037::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0378757 s, 27.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:37:47,573::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:37:48,050::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:48,136::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0438544 s, 23.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:37:48,434::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:37:48,444::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00168562 s, 2.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:37:48,588::task::595::TaskManager.Task::(_updateState) Task=`6c1e0c2d-64b9-497e-bfb2-9aa84753dacb`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:37:48,588::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:37:48,589::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00168562', 'lastCheck': '0.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:37:48,589::task::1185::TaskManager.Task::(prepare) Task=`6c1e0c2d-64b9-497e-bfb2-9aa84753dacb`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00168562', 'lastCheck': '0.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:37:48,589::task::595::TaskManager.Task::(_updateState) Task=`6c1e0c2d-64b9-497e-bfb2-9aa84753dacb`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:37:48,589::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:37:48,589::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:37:48,589::task::990::TaskManager.Task::(_decref) Task=`6c1e0c2d-64b9-497e-bfb2-9aa84753dacb`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:37:50,149::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:50,243::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374921 s, 27.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:37:50,441::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:37:50,441::task::595::TaskManager.Task::(_updateState) Task=`48b19abb-079e-4cd8-8a0b-092d28243dcb`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:37:50,442::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:37:50,442::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:37:50,442::task::1185::TaskManager.Task::(prepare) Task=`48b19abb-079e-4cd8-8a0b-092d28243dcb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:37:50,442::task::595::TaskManager.Task::(_updateState) Task=`48b19abb-079e-4cd8-8a0b-092d28243dcb`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:37:50,443::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:37:50,443::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:37:50,443::task::990::TaskManager.Task::(_decref) Task=`48b19abb-079e-4cd8-8a0b-092d28243dcb`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:37:50,448::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:37:50,448::task::595::TaskManager.Task::(_updateState) Task=`dcad9460-fd31-4bd2-b9c6-464c37e0fd84`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:37:50,448::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:37:50,449::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`dc0b8857-0899-4e73-b4d6-c44b423ae143`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:37:50,449::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:37:50,449::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:37:50,449::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`dc0b8857-0899-4e73-b4d6-c44b423ae143`::Granted request >Thread-338::DEBUG::2014-05-13 11:37:50,450::task::827::TaskManager.Task::(resourceAcquired) Task=`dcad9460-fd31-4bd2-b9c6-464c37e0fd84`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:37:50,450::task::990::TaskManager.Task::(_decref) Task=`dcad9460-fd31-4bd2-b9c6-464c37e0fd84`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:37:50,450::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:37:50,451::task::1185::TaskManager.Task::(prepare) Task=`dcad9460-fd31-4bd2-b9c6-464c37e0fd84`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:37:50,451::task::595::TaskManager.Task::(_updateState) Task=`dcad9460-fd31-4bd2-b9c6-464c37e0fd84`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:37:50,451::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:37:50,451::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:37:50,451::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:37:50,452::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:37:50,452::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:37:50,452::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:37:50,452::task::990::TaskManager.Task::(_decref) Task=`dcad9460-fd31-4bd2-b9c6-464c37e0fd84`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:37:50,594::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:37:52,256::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:52,330::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0326235 s, 31.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:37:53,614::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:37:54,343::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:54,418::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323227 s, 31.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:37:56,432::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:56,512::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037633 s, 27.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:37:56,631::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:37:58,450::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:37:58,459::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000664595 s, 6.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:37:58,525::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:37:58,606::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0368174 s, 27.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:37:59,652::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:38:00,471::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:38:00,472::task::595::TaskManager.Task::(_updateState) Task=`2f692851-b5c4-4089-880c-732321e413e0`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:38:00,472::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:38:00,472::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:38:00,472::task::1185::TaskManager.Task::(prepare) Task=`2f692851-b5c4-4089-880c-732321e413e0`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:38:00,473::task::595::TaskManager.Task::(_updateState) Task=`2f692851-b5c4-4089-880c-732321e413e0`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:38:00,473::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:38:00,473::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:38:00,473::task::990::TaskManager.Task::(_decref) Task=`2f692851-b5c4-4089-880c-732321e413e0`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:38:00,480::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:38:00,480::task::595::TaskManager.Task::(_updateState) Task=`76cf2443-72b9-48ed-89d5-cdb0db4a5715`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:38:00,481::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:38:00,481::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fc9cb724-ce29-4e3d-a29d-15fcefdc4e15`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:38:00,481::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:38:00,482::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:38:00,482::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fc9cb724-ce29-4e3d-a29d-15fcefdc4e15`::Granted request >Thread-338::DEBUG::2014-05-13 11:38:00,482::task::827::TaskManager.Task::(resourceAcquired) Task=`76cf2443-72b9-48ed-89d5-cdb0db4a5715`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:38:00,482::task::990::TaskManager.Task::(_decref) Task=`76cf2443-72b9-48ed-89d5-cdb0db4a5715`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:38:00,483::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:38:00,483::task::1185::TaskManager.Task::(prepare) Task=`76cf2443-72b9-48ed-89d5-cdb0db4a5715`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:38:00,483::task::595::TaskManager.Task::(_updateState) Task=`76cf2443-72b9-48ed-89d5-cdb0db4a5715`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:38:00,484::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:38:00,484::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:38:00,484::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:38:00,484::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:38:00,484::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:38:00,484::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:38:00,485::task::990::TaskManager.Task::(_decref) Task=`76cf2443-72b9-48ed-89d5-cdb0db4a5715`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:38:00,619::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:00,995::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.33138 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:38:02,669::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:38:03,008::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:03,083::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0328076 s, 31.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:38:03,730::task::595::TaskManager.Task::(_updateState) Task=`9042d5b7-7931-4591-b839-8146990d8405`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:38:03,730::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:38:03,730::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000664595', 'lastCheck': '5.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:38:03,730::task::1185::TaskManager.Task::(prepare) Task=`9042d5b7-7931-4591-b839-8146990d8405`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000664595', 'lastCheck': '5.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:38:03,731::task::595::TaskManager.Task::(_updateState) Task=`9042d5b7-7931-4591-b839-8146990d8405`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:38:03,731::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:38:03,731::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:38:03,731::task::990::TaskManager.Task::(_decref) Task=`9042d5b7-7931-4591-b839-8146990d8405`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:38:05,096::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:05,177::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0380842 s, 26.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:38:05,686::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:38:07,189::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:07,265::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0326301 s, 31.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:38:08,466::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:38:08,475::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000958566 s, 4.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:38:08,708::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:38:09,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:09,353::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0330161 s, 31.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:38:10,505::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2ec76adc] >Thread-13::DEBUG::2014-05-13 11:38:10,505::task::595::TaskManager.Task::(_updateState) Task=`c16692a8-d82c-4928-ba79-07a5869e4dff`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:38:10,505::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:38:10,506::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:38:10,506::task::1185::TaskManager.Task::(prepare) Task=`c16692a8-d82c-4928-ba79-07a5869e4dff`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:38:10,506::task::595::TaskManager.Task::(_updateState) Task=`c16692a8-d82c-4928-ba79-07a5869e4dff`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:38:10,506::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:38:10,506::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:38:10,507::task::990::TaskManager.Task::(_decref) Task=`c16692a8-d82c-4928-ba79-07a5869e4dff`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:38:10,513::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2ec76adc] >Thread-338::DEBUG::2014-05-13 11:38:10,514::task::595::TaskManager.Task::(_updateState) Task=`e03e61dc-16fc-4616-ace3-5223c7f24a7a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:38:10,514::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:38:10,514::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5f469c4c-2cbe-4e7a-a1a7-63e0b2165bc7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:38:10,514::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:38:10,515::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:38:10,515::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5f469c4c-2cbe-4e7a-a1a7-63e0b2165bc7`::Granted request >Thread-338::DEBUG::2014-05-13 11:38:10,515::task::827::TaskManager.Task::(resourceAcquired) Task=`e03e61dc-16fc-4616-ace3-5223c7f24a7a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:38:10,515::task::990::TaskManager.Task::(_decref) Task=`e03e61dc-16fc-4616-ace3-5223c7f24a7a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:38:10,516::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:38:10,516::task::1185::TaskManager.Task::(prepare) Task=`e03e61dc-16fc-4616-ace3-5223c7f24a7a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:38:10,517::task::595::TaskManager.Task::(_updateState) Task=`e03e61dc-16fc-4616-ace3-5223c7f24a7a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:38:10,517::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:38:10,517::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:38:10,517::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:38:10,517::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:38:10,517::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:38:10,518::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:38:10,518::task::990::TaskManager.Task::(_decref) Task=`e03e61dc-16fc-4616-ace3-5223c7f24a7a`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:38:11,367::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:11,447::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0375602 s, 27.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:38:11,726::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:38:13,460::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:13,541::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.039446 s, 26.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:38:14,744::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:38:15,554::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:15,648::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0517556 s, 19.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:38:17,661::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:17,737::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0336304 s, 30.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:38:17,762::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:38:18,482::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:38:18,491::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000652945 s, 6.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:38:18,887::task::595::TaskManager.Task::(_updateState) Task=`081f2b64-35c9-47ee-a729-a2635567893e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:38:18,888::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:38:18,888::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000652945', 'lastCheck': '0.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:38:18,888::task::1185::TaskManager.Task::(prepare) Task=`081f2b64-35c9-47ee-a729-a2635567893e`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000652945', 'lastCheck': '0.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:38:18,888::task::595::TaskManager.Task::(_updateState) Task=`081f2b64-35c9-47ee-a729-a2635567893e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:38:18,888::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:38:18,889::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:38:18,889::task::990::TaskManager.Task::(_decref) Task=`081f2b64-35c9-47ee-a729-a2635567893e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:38:19,750::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:19,829::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356157 s, 28.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:38:20,537::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5affc0d0] >Thread-13::DEBUG::2014-05-13 11:38:20,538::task::595::TaskManager.Task::(_updateState) Task=`df1ec5e1-dedf-4190-95ab-875de3e72883`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:38:20,538::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:38:20,538::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:38:20,538::task::1185::TaskManager.Task::(prepare) Task=`df1ec5e1-dedf-4190-95ab-875de3e72883`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:38:20,539::task::595::TaskManager.Task::(_updateState) Task=`df1ec5e1-dedf-4190-95ab-875de3e72883`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:38:20,539::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:38:20,539::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:38:20,539::task::990::TaskManager.Task::(_decref) Task=`df1ec5e1-dedf-4190-95ab-875de3e72883`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:38:20,544::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5affc0d0] >Thread-338::DEBUG::2014-05-13 11:38:20,545::task::595::TaskManager.Task::(_updateState) Task=`105f33fe-d8bb-49e0-b083-104c666a1eb3`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:38:20,545::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:38:20,545::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`986d1f9c-1306-4d6a-9013-65c65e91f306`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:38:20,546::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:38:20,547::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:38:20,547::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`986d1f9c-1306-4d6a-9013-65c65e91f306`::Granted request >Thread-338::DEBUG::2014-05-13 11:38:20,547::task::827::TaskManager.Task::(resourceAcquired) Task=`105f33fe-d8bb-49e0-b083-104c666a1eb3`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:38:20,547::task::990::TaskManager.Task::(_decref) Task=`105f33fe-d8bb-49e0-b083-104c666a1eb3`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:38:20,548::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:38:20,548::task::1185::TaskManager.Task::(prepare) Task=`105f33fe-d8bb-49e0-b083-104c666a1eb3`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:38:20,549::task::595::TaskManager.Task::(_updateState) Task=`105f33fe-d8bb-49e0-b083-104c666a1eb3`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:38:20,549::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:38:20,549::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:38:20,549::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:38:20,549::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:38:20,549::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:38:20,550::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:38:20,550::task::990::TaskManager.Task::(_decref) Task=`105f33fe-d8bb-49e0-b083-104c666a1eb3`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:38:20,778::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:38:21,842::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:21,915::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0302299 s, 33.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:38:23,797::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:38:23,928::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:24,002::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303002 s, 33.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:38:26,015::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:26,093::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0354578 s, 28.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:38:26,821::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:38:28,106::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:28,179::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310123 s, 33.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:38:28,498::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:38:28,507::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000681283 s, 6.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:38:29,839::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:38:30,192::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:30,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.045337 s, 22.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:38:30,573::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [dc1f215] >Thread-13::DEBUG::2014-05-13 11:38:30,573::task::595::TaskManager.Task::(_updateState) Task=`aa5158ca-3023-44c8-a9af-297e890f7bfb`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:38:30,574::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:38:30,574::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:38:30,574::task::1185::TaskManager.Task::(prepare) Task=`aa5158ca-3023-44c8-a9af-297e890f7bfb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:38:30,574::task::595::TaskManager.Task::(_updateState) Task=`aa5158ca-3023-44c8-a9af-297e890f7bfb`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:38:30,575::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:38:30,575::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:38:30,575::task::990::TaskManager.Task::(_decref) Task=`aa5158ca-3023-44c8-a9af-297e890f7bfb`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:38:30,580::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [dc1f215] >Thread-338::DEBUG::2014-05-13 11:38:30,580::task::595::TaskManager.Task::(_updateState) Task=`2698dd5c-70ea-4080-9b00-8b56f097ad6f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:38:30,580::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:38:30,581::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a73e3188-4291-4dce-a56d-dacd8e201e20`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:38:30,581::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:38:30,581::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:38:30,581::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a73e3188-4291-4dce-a56d-dacd8e201e20`::Granted request >Thread-338::DEBUG::2014-05-13 11:38:30,582::task::827::TaskManager.Task::(resourceAcquired) Task=`2698dd5c-70ea-4080-9b00-8b56f097ad6f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:38:30,582::task::990::TaskManager.Task::(_decref) Task=`2698dd5c-70ea-4080-9b00-8b56f097ad6f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:38:30,582::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:38:30,583::task::1185::TaskManager.Task::(prepare) Task=`2698dd5c-70ea-4080-9b00-8b56f097ad6f`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:38:30,583::task::595::TaskManager.Task::(_updateState) Task=`2698dd5c-70ea-4080-9b00-8b56f097ad6f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:38:30,583::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:38:30,583::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:38:30,583::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:38:30,584::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:38:30,584::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:38:30,584::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:38:30,584::task::990::TaskManager.Task::(_decref) Task=`2698dd5c-70ea-4080-9b00-8b56f097ad6f`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:38:32,292::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:32,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0504539 s, 20.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:38:32,858::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:38:34,027::task::595::TaskManager.Task::(_updateState) Task=`509a408e-80ea-47b2-8c8d-b720903f7ca1`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:38:34,027::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:38:34,027::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000681283', 'lastCheck': '5.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:38:34,028::task::1185::TaskManager.Task::(prepare) Task=`509a408e-80ea-47b2-8c8d-b720903f7ca1`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000681283', 'lastCheck': '5.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:38:34,028::task::595::TaskManager.Task::(_updateState) Task=`509a408e-80ea-47b2-8c8d-b720903f7ca1`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:38:34,028::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:38:34,028::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:38:34,028::task::990::TaskManager.Task::(_decref) Task=`509a408e-80ea-47b2-8c8d-b720903f7ca1`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:38:34,399::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:34,476::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0337337 s, 30.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:38:35,877::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:38:36,489::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:36,563::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319402 s, 32.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:38:38,514::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:38:38,523::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000716459 s, 5.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:38:38,574::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:38,652::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0348798 s, 29.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:38:38,894::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:38:40,605::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3c71a9dd] >Thread-13::DEBUG::2014-05-13 11:38:40,605::task::595::TaskManager.Task::(_updateState) Task=`f1e059be-c318-4c7f-a166-c0bf6db8aa09`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:38:40,605::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:38:40,606::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:38:40,606::task::1185::TaskManager.Task::(prepare) Task=`f1e059be-c318-4c7f-a166-c0bf6db8aa09`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:38:40,606::task::595::TaskManager.Task::(_updateState) Task=`f1e059be-c318-4c7f-a166-c0bf6db8aa09`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:38:40,606::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:38:40,606::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:38:40,607::task::990::TaskManager.Task::(_decref) Task=`f1e059be-c318-4c7f-a166-c0bf6db8aa09`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:38:40,611::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3c71a9dd] >Thread-338::DEBUG::2014-05-13 11:38:40,611::task::595::TaskManager.Task::(_updateState) Task=`8be822fa-6b05-41ac-8513-11f22ebc5b9f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:38:40,612::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:38:40,612::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`20f5d36b-52b8-4557-8e89-051593e8b99f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:38:40,612::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:38:40,612::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:38:40,613::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`20f5d36b-52b8-4557-8e89-051593e8b99f`::Granted request >Thread-338::DEBUG::2014-05-13 11:38:40,613::task::827::TaskManager.Task::(resourceAcquired) Task=`8be822fa-6b05-41ac-8513-11f22ebc5b9f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:38:40,613::task::990::TaskManager.Task::(_decref) Task=`8be822fa-6b05-41ac-8513-11f22ebc5b9f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:38:40,614::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:38:40,614::task::1185::TaskManager.Task::(prepare) Task=`8be822fa-6b05-41ac-8513-11f22ebc5b9f`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:38:40,614::task::595::TaskManager.Task::(_updateState) Task=`8be822fa-6b05-41ac-8513-11f22ebc5b9f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:38:40,614::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:38:40,615::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:38:40,615::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:38:40,615::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:38:40,615::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:38:40,615::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:38:40,616::task::990::TaskManager.Task::(_decref) Task=`8be822fa-6b05-41ac-8513-11f22ebc5b9f`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:38:40,665::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:40,747::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0375356 s, 27.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:38:41,913::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:38:42,760::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:43,175::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.367424 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:38:44,930::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:38:45,188::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:45,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353153 s, 29.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:38:47,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:47,359::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0371241 s, 27.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:38:47,948::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:38:48,529::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:38:48,538::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000732604 s, 5.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:38:49,164::task::595::TaskManager.Task::(_updateState) Task=`377f9641-4f75-4de8-acfa-61523c7c8131`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:38:49,164::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:38:49,164::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000732604', 'lastCheck': '0.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:38:49,164::task::1185::TaskManager.Task::(prepare) Task=`377f9641-4f75-4de8-acfa-61523c7c8131`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000732604', 'lastCheck': '0.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:38:49,165::task::595::TaskManager.Task::(_updateState) Task=`377f9641-4f75-4de8-acfa-61523c7c8131`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:38:49,165::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:38:49,165::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:38:49,165::task::990::TaskManager.Task::(_decref) Task=`377f9641-4f75-4de8-acfa-61523c7c8131`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:38:49,372::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:49,450::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035592 s, 28.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:38:50,648::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:38:50,648::task::595::TaskManager.Task::(_updateState) Task=`79e151c2-7029-4345-b0a9-31fe755e9840`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:38:50,649::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:38:50,649::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:38:50,649::task::1185::TaskManager.Task::(prepare) Task=`79e151c2-7029-4345-b0a9-31fe755e9840`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:38:50,649::task::595::TaskManager.Task::(_updateState) Task=`79e151c2-7029-4345-b0a9-31fe755e9840`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:38:50,649::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:38:50,650::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:38:50,650::task::990::TaskManager.Task::(_decref) Task=`79e151c2-7029-4345-b0a9-31fe755e9840`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:38:50,655::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:38:50,655::task::595::TaskManager.Task::(_updateState) Task=`2b26fd85-7398-4453-a14a-4d22c3b508eb`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:38:50,655::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:38:50,656::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`359701cf-734a-4931-92e5-edf549d12e66`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:38:50,656::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:38:50,656::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:38:50,656::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`359701cf-734a-4931-92e5-edf549d12e66`::Granted request >Thread-338::DEBUG::2014-05-13 11:38:50,657::task::827::TaskManager.Task::(resourceAcquired) Task=`2b26fd85-7398-4453-a14a-4d22c3b508eb`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:38:50,657::task::990::TaskManager.Task::(_decref) Task=`2b26fd85-7398-4453-a14a-4d22c3b508eb`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:38:50,657::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:38:50,658::task::1185::TaskManager.Task::(prepare) Task=`2b26fd85-7398-4453-a14a-4d22c3b508eb`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:38:50,658::task::595::TaskManager.Task::(_updateState) Task=`2b26fd85-7398-4453-a14a-4d22c3b508eb`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:38:50,658::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:38:50,658::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:38:50,658::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:38:50,659::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:38:50,659::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:38:50,659::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:38:50,659::task::990::TaskManager.Task::(_decref) Task=`2b26fd85-7398-4453-a14a-4d22c3b508eb`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:38:50,966::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:38:51,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:51,544::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365656 s, 28.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:38:53,557::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:53,756::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.120681 s, 8.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:38:53,984::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:38:55,764::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:55,854::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0459926 s, 22.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:38:57,003::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:38:57,867::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:38:57,940::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0299622 s, 34.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:38:58,545::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:38:58,553::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000641797 s, 6.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:38:59,953::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:39:00,026::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:39:00,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313933 s, 32.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:39:00,678::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:39:00,679::task::595::TaskManager.Task::(_updateState) Task=`80c438c8-4d1b-4bd3-8e06-66d5cac91055`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:39:00,679::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:39:00,680::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:39:00,680::task::1185::TaskManager.Task::(prepare) Task=`80c438c8-4d1b-4bd3-8e06-66d5cac91055`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:39:00,680::task::595::TaskManager.Task::(_updateState) Task=`80c438c8-4d1b-4bd3-8e06-66d5cac91055`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:39:00,680::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:39:00,680::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:39:00,680::task::990::TaskManager.Task::(_decref) Task=`80c438c8-4d1b-4bd3-8e06-66d5cac91055`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:39:00,685::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:39:00,685::task::595::TaskManager.Task::(_updateState) Task=`9ce01e88-68a0-48ec-90cc-1e02819dfad5`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:39:00,686::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:39:00,686::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`157040eb-89b3-4414-b83c-7422c5ca87d0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:39:00,686::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:39:00,686::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:39:00,687::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`157040eb-89b3-4414-b83c-7422c5ca87d0`::Granted request >Thread-338::DEBUG::2014-05-13 11:39:00,687::task::827::TaskManager.Task::(resourceAcquired) Task=`9ce01e88-68a0-48ec-90cc-1e02819dfad5`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:39:00,687::task::990::TaskManager.Task::(_decref) Task=`9ce01e88-68a0-48ec-90cc-1e02819dfad5`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:39:00,688::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:39:00,688::task::1185::TaskManager.Task::(prepare) Task=`9ce01e88-68a0-48ec-90cc-1e02819dfad5`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:39:00,688::task::595::TaskManager.Task::(_updateState) Task=`9ce01e88-68a0-48ec-90cc-1e02819dfad5`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:39:00,688::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:39:00,689::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:39:00,689::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:39:00,689::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:39:00,689::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:39:00,689::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:39:00,690::task::990::TaskManager.Task::(_decref) Task=`9ce01e88-68a0-48ec-90cc-1e02819dfad5`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:39:02,045::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:02,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0366699 s, 27.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:39:03,045::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:39:04,139::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:04,214::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321295 s, 31.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:39:04,301::task::595::TaskManager.Task::(_updateState) Task=`a2d25e39-2faf-4620-ade8-b45e85e31424`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:39:04,301::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:39:04,301::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000641797', 'lastCheck': '5.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:39:04,302::task::1185::TaskManager.Task::(prepare) Task=`a2d25e39-2faf-4620-ade8-b45e85e31424`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000641797', 'lastCheck': '5.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:39:04,302::task::595::TaskManager.Task::(_updateState) Task=`a2d25e39-2faf-4620-ade8-b45e85e31424`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:39:04,302::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:39:04,302::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:39:04,302::task::990::TaskManager.Task::(_decref) Task=`a2d25e39-2faf-4620-ade8-b45e85e31424`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:39:06,066::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:39:06,227::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:06,303::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.03137 s, 32.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:39:08,316::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:08,394::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0349828 s, 29.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:39:08,560::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:39:08,568::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000633419 s, 6.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:39:09,084::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:39:10,407::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:10,488::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382533 s, 26.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:39:10,709::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6019f20] >Thread-13::DEBUG::2014-05-13 11:39:10,709::task::595::TaskManager.Task::(_updateState) Task=`a677efed-97ce-4d52-8b08-d32e32a18e7e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:39:10,710::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:39:10,710::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:39:10,710::task::1185::TaskManager.Task::(prepare) Task=`a677efed-97ce-4d52-8b08-d32e32a18e7e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:39:10,710::task::595::TaskManager.Task::(_updateState) Task=`a677efed-97ce-4d52-8b08-d32e32a18e7e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:39:10,711::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:39:10,711::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:39:10,711::task::990::TaskManager.Task::(_decref) Task=`a677efed-97ce-4d52-8b08-d32e32a18e7e`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:39:10,715::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6019f20] >Thread-338::DEBUG::2014-05-13 11:39:10,716::task::595::TaskManager.Task::(_updateState) Task=`f7a36c83-00f4-4775-9fed-80b19425da76`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:39:10,716::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:39:10,716::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b2faabda-beff-4d76-9b18-592b30a09c34`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:39:10,717::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:39:10,717::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:39:10,717::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b2faabda-beff-4d76-9b18-592b30a09c34`::Granted request >Thread-338::DEBUG::2014-05-13 11:39:10,717::task::827::TaskManager.Task::(resourceAcquired) Task=`f7a36c83-00f4-4775-9fed-80b19425da76`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:39:10,718::task::990::TaskManager.Task::(_decref) Task=`f7a36c83-00f4-4775-9fed-80b19425da76`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:39:10,718::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:39:10,718::task::1185::TaskManager.Task::(prepare) Task=`f7a36c83-00f4-4775-9fed-80b19425da76`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:39:10,719::task::595::TaskManager.Task::(_updateState) Task=`f7a36c83-00f4-4775-9fed-80b19425da76`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:39:10,719::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:39:10,719::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:39:10,719::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:39:10,719::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:39:10,720::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:39:10,720::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:39:10,720::task::990::TaskManager.Task::(_decref) Task=`f7a36c83-00f4-4775-9fed-80b19425da76`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:39:12,106::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:39:12,501::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:12,590::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0446419 s, 22.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:39:14,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:14,682::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362181 s, 28.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:39:15,127::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:39:16,694::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:16,774::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373888 s, 27.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:39:18,147::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:39:18,575::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:39:18,587::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00384147 s, 1.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:39:18,787::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:18,866::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036045 s, 28.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:39:19,493::task::595::TaskManager.Task::(_updateState) Task=`f62e6f59-d8a0-4f3f-8f3a-8a308fe80376`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:39:19,493::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:39:19,493::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00384147', 'lastCheck': '0.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:39:19,494::task::1185::TaskManager.Task::(prepare) Task=`f62e6f59-d8a0-4f3f-8f3a-8a308fe80376`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00384147', 'lastCheck': '0.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:39:19,494::task::595::TaskManager.Task::(_updateState) Task=`f62e6f59-d8a0-4f3f-8f3a-8a308fe80376`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:39:19,494::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:39:19,494::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:39:19,494::task::990::TaskManager.Task::(_decref) Task=`f62e6f59-d8a0-4f3f-8f3a-8a308fe80376`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 11:39:20,739::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:39:20,739::task::595::TaskManager.Task::(_updateState) Task=`68415207-c61f-4d12-94ca-ca98a47a1c93`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:39:20,739::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:39:20,740::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:39:20,740::task::1185::TaskManager.Task::(prepare) Task=`68415207-c61f-4d12-94ca-ca98a47a1c93`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:39:20,740::task::595::TaskManager.Task::(_updateState) Task=`68415207-c61f-4d12-94ca-ca98a47a1c93`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:39:20,740::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:39:20,740::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:39:20,741::task::990::TaskManager.Task::(_decref) Task=`68415207-c61f-4d12-94ca-ca98a47a1c93`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:39:20,745::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:39:20,746::task::595::TaskManager.Task::(_updateState) Task=`7b66ad64-ceb6-4cc3-a97a-bc74f92d7564`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:39:20,746::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:39:20,746::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`202ff82a-d4be-4154-a254-7eea67da6cdd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:39:20,747::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:39:20,747::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:39:20,747::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`202ff82a-d4be-4154-a254-7eea67da6cdd`::Granted request >Thread-338::DEBUG::2014-05-13 11:39:20,747::task::827::TaskManager.Task::(resourceAcquired) Task=`7b66ad64-ceb6-4cc3-a97a-bc74f92d7564`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:39:20,747::task::990::TaskManager.Task::(_decref) Task=`7b66ad64-ceb6-4cc3-a97a-bc74f92d7564`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:39:20,748::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:39:20,748::task::1185::TaskManager.Task::(prepare) Task=`7b66ad64-ceb6-4cc3-a97a-bc74f92d7564`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:39:20,749::task::595::TaskManager.Task::(_updateState) Task=`7b66ad64-ceb6-4cc3-a97a-bc74f92d7564`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:39:20,749::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:39:20,749::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:39:20,749::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:39:20,749::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:39:20,750::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:39:20,750::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:39:20,750::task::990::TaskManager.Task::(_decref) Task=`7b66ad64-ceb6-4cc3-a97a-bc74f92d7564`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:39:20,879::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:20,954::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315545 s, 32.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:39:21,165::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:39:22,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:23,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379877 s, 27.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:39:24,183::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:39:25,061::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:25,147::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0429742 s, 23.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:39:27,160::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:39:27,203::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:39:27,237::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307152 s, 33.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:39:28,594::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:39:28,606::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000557803 s, 7.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:39:29,250::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:29,328::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353757 s, 28.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:39:30,222::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:39:30,768::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3256e43f] >Thread-13::DEBUG::2014-05-13 11:39:30,768::task::595::TaskManager.Task::(_updateState) Task=`f37434b2-a336-44e8-912e-5634289a001e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:39:30,768::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:39:30,769::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:39:30,769::task::1185::TaskManager.Task::(prepare) Task=`f37434b2-a336-44e8-912e-5634289a001e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:39:30,769::task::595::TaskManager.Task::(_updateState) Task=`f37434b2-a336-44e8-912e-5634289a001e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:39:30,769::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:39:30,770::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:39:30,770::task::990::TaskManager.Task::(_decref) Task=`f37434b2-a336-44e8-912e-5634289a001e`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:39:30,774::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3256e43f] >Thread-338::DEBUG::2014-05-13 11:39:30,774::task::595::TaskManager.Task::(_updateState) Task=`507c5468-1fb0-4ccb-a24e-2c22bfb11870`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:39:30,775::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:39:30,775::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`33fc6dfa-b62a-4310-a20c-011adce5eecd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:39:30,775::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:39:30,775::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:39:30,776::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`33fc6dfa-b62a-4310-a20c-011adce5eecd`::Granted request >Thread-338::DEBUG::2014-05-13 11:39:30,777::task::827::TaskManager.Task::(resourceAcquired) Task=`507c5468-1fb0-4ccb-a24e-2c22bfb11870`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:39:30,777::task::990::TaskManager.Task::(_decref) Task=`507c5468-1fb0-4ccb-a24e-2c22bfb11870`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:39:30,778::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:39:30,778::task::1185::TaskManager.Task::(prepare) Task=`507c5468-1fb0-4ccb-a24e-2c22bfb11870`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:39:30,778::task::595::TaskManager.Task::(_updateState) Task=`507c5468-1fb0-4ccb-a24e-2c22bfb11870`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:39:30,778::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:39:30,778::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:39:30,779::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:39:30,779::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:39:30,779::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:39:30,779::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:39:30,779::task::990::TaskManager.Task::(_decref) Task=`507c5468-1fb0-4ccb-a24e-2c22bfb11870`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:39:31,341::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:31,422::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0378323 s, 27.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:39:33,244::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:39:33,435::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:33,519::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387523 s, 26.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:39:34,631::task::595::TaskManager.Task::(_updateState) Task=`8a9d0d79-a82f-40ca-9951-1641773f7355`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:39:34,632::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:39:34,632::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000557803', 'lastCheck': '6.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:39:34,632::task::1185::TaskManager.Task::(prepare) Task=`8a9d0d79-a82f-40ca-9951-1641773f7355`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000557803', 'lastCheck': '6.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:39:34,632::task::595::TaskManager.Task::(_updateState) Task=`8a9d0d79-a82f-40ca-9951-1641773f7355`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:39:34,632::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:39:34,633::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:39:34,633::task::990::TaskManager.Task::(_decref) Task=`8a9d0d79-a82f-40ca-9951-1641773f7355`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:39:35,532::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:35,611::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352463 s, 29.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:39:36,263::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:39:37,624::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:37,990::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.323018 s, 3.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:39:38,613::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:39:38,622::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00074388 s, 5.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:39:39,282::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:39:40,002::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:40,076::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0301099 s, 34.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:39:40,803::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:39:40,804::task::595::TaskManager.Task::(_updateState) Task=`b3d21e89-d4cd-4ee1-8cd3-515818bde189`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:39:40,804::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:39:40,804::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:39:40,804::task::1185::TaskManager.Task::(prepare) Task=`b3d21e89-d4cd-4ee1-8cd3-515818bde189`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:39:40,804::task::595::TaskManager.Task::(_updateState) Task=`b3d21e89-d4cd-4ee1-8cd3-515818bde189`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:39:40,805::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:39:40,805::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:39:40,805::task::990::TaskManager.Task::(_decref) Task=`b3d21e89-d4cd-4ee1-8cd3-515818bde189`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:39:40,812::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:39:40,812::task::595::TaskManager.Task::(_updateState) Task=`c6010fd9-b3ec-4821-bba8-bbe5b27b6539`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:39:40,812::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:39:40,813::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`287458e1-7e47-4b2c-a602-863cbc964a08`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:39:40,813::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:39:40,813::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:39:40,814::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`287458e1-7e47-4b2c-a602-863cbc964a08`::Granted request >Thread-338::DEBUG::2014-05-13 11:39:40,814::task::827::TaskManager.Task::(resourceAcquired) Task=`c6010fd9-b3ec-4821-bba8-bbe5b27b6539`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:39:40,814::task::990::TaskManager.Task::(_decref) Task=`c6010fd9-b3ec-4821-bba8-bbe5b27b6539`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:39:40,815::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:39:40,815::task::1185::TaskManager.Task::(prepare) Task=`c6010fd9-b3ec-4821-bba8-bbe5b27b6539`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:39:40,815::task::595::TaskManager.Task::(_updateState) Task=`c6010fd9-b3ec-4821-bba8-bbe5b27b6539`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:39:40,815::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:39:40,816::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:39:40,816::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:39:40,816::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:39:40,816::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:39:40,816::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:39:40,817::task::990::TaskManager.Task::(_decref) Task=`c6010fd9-b3ec-4821-bba8-bbe5b27b6539`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:39:42,089::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:42,162::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314416 s, 32.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:39:42,299::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:39:44,175::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:44,254::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356378 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:39:45,318::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:39:46,267::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:46,347::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359232 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:39:48,336::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:39:48,360::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:48,435::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312482 s, 32.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:39:48,629::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:39:48,637::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00067988 s, 6.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:39:49,767::task::595::TaskManager.Task::(_updateState) Task=`f6426526-82f1-4ff0-b707-eecf1a0f4074`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:39:49,767::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:39:49,767::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00067988', 'lastCheck': '1.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:39:49,768::task::1185::TaskManager.Task::(prepare) Task=`f6426526-82f1-4ff0-b707-eecf1a0f4074`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00067988', 'lastCheck': '1.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:39:49,768::task::595::TaskManager.Task::(_updateState) Task=`f6426526-82f1-4ff0-b707-eecf1a0f4074`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:39:49,768::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:39:49,768::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:39:49,768::task::990::TaskManager.Task::(_decref) Task=`f6426526-82f1-4ff0-b707-eecf1a0f4074`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:39:50,448::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:50,542::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0385334 s, 26.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:39:50,837::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5affc0d0] >Thread-13::DEBUG::2014-05-13 11:39:50,838::task::595::TaskManager.Task::(_updateState) Task=`95f44ae8-780b-4395-b9d4-dc519b48292d`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:39:50,838::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:39:50,838::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:39:50,838::task::1185::TaskManager.Task::(prepare) Task=`95f44ae8-780b-4395-b9d4-dc519b48292d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:39:50,839::task::595::TaskManager.Task::(_updateState) Task=`95f44ae8-780b-4395-b9d4-dc519b48292d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:39:50,839::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:39:50,839::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:39:50,839::task::990::TaskManager.Task::(_decref) Task=`95f44ae8-780b-4395-b9d4-dc519b48292d`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:39:50,844::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5affc0d0] >Thread-338::DEBUG::2014-05-13 11:39:50,844::task::595::TaskManager.Task::(_updateState) Task=`9a59535b-1b49-4f2a-8d40-c125aea2ddf2`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:39:50,844::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:39:50,845::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`82115d15-9f20-473e-b5a3-6b80908d6e79`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:39:50,845::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:39:50,845::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:39:50,845::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`82115d15-9f20-473e-b5a3-6b80908d6e79`::Granted request >Thread-338::DEBUG::2014-05-13 11:39:50,845::task::827::TaskManager.Task::(resourceAcquired) Task=`9a59535b-1b49-4f2a-8d40-c125aea2ddf2`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:39:50,846::task::990::TaskManager.Task::(_decref) Task=`9a59535b-1b49-4f2a-8d40-c125aea2ddf2`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:39:50,846::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:39:50,846::task::1185::TaskManager.Task::(prepare) Task=`9a59535b-1b49-4f2a-8d40-c125aea2ddf2`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:39:50,847::task::595::TaskManager.Task::(_updateState) Task=`9a59535b-1b49-4f2a-8d40-c125aea2ddf2`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:39:50,847::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:39:50,847::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:39:50,847::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:39:50,847::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:39:50,847::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:39:50,847::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:39:50,848::task::990::TaskManager.Task::(_decref) Task=`9a59535b-1b49-4f2a-8d40-c125aea2ddf2`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:39:51,354::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:39:52,555::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:52,641::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0443453 s, 23.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:39:54,379::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:39:54,654::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:54,732::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.033212 s, 30.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:39:56,745::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:56,818::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035151 s, 29.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:39:57,397::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:39:58,642::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 41830dad-f012-4b4d-b636-863f66d5ca4b >Thread-322::DEBUG::2014-05-13 11:39:58,642::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-322::DEBUG::2014-05-13 11:39:58,645::blockSD::336::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:39:58,653::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000733458 s, 2.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:39:58,654::misc::287::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000733458 s, 2.8 MB/s'], size: 2048 >Thread-322::DEBUG::2014-05-13 11:39:58,654::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-322::WARNING::2014-05-13 11:39:58,654::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-322::DEBUG::2014-05-13 11:39:58,654::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-322::DEBUG::2014-05-13 11:39:58,655::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:39:58,655::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:39:58,655::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:39:58,655::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:39:58,656::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:39:58,657::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:58,826::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:39:58,913::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362026 s, 28.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:39:58,939::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:39:58,941::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:39:58,941::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'LOCKPOLICY=', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', 'DESCRIPTION=ISCSI', 'LOGBLKSIZE=512', 'TYPE=ISCSI', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'PHYBLKSIZE=512', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', 'CLASS=Data', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'POOL_DESCRIPTION=DC_2', 'POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'MASTER_VERSION=1', 'ROLE=Master', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', '_SHA_CKSUM=28bb2d7cc60e0d779eb69fe93d050c16c769f069', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0'] >Thread-322::INFO::2014-05-13 11:39:58,942::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_imageNS already registered >Thread-322::INFO::2014-05-13 11:39:58,942::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_volumeNS already registered >Thread-322::INFO::2014-05-13 11:39:58,943::blockSD::452::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_lvmActivationNS already registered >Thread-322::DEBUG::2014-05-13 11:39:58,943::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 11:39:59,152::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:39:59,155::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:39:59,164::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000678332 s, 6.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:40:00,417::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:40:00,865::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [dc1f215] >Thread-13::DEBUG::2014-05-13 11:40:00,866::task::595::TaskManager.Task::(_updateState) Task=`bb93fa69-190e-4175-9513-2b8c224e3a94`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:40:00,866::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:40:00,866::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:40:00,867::task::1185::TaskManager.Task::(prepare) Task=`bb93fa69-190e-4175-9513-2b8c224e3a94`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:40:00,867::task::595::TaskManager.Task::(_updateState) Task=`bb93fa69-190e-4175-9513-2b8c224e3a94`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:40:00,867::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:40:00,867::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:40:00,867::task::990::TaskManager.Task::(_decref) Task=`bb93fa69-190e-4175-9513-2b8c224e3a94`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:40:00,874::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [dc1f215] >Thread-338::DEBUG::2014-05-13 11:40:00,874::task::595::TaskManager.Task::(_updateState) Task=`bfff2103-106a-4cf0-983b-eaf8eff7ad83`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:40:00,875::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:40:00,875::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`78fe4911-5f73-4fe9-91e7-b226b808964a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:40:00,875::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:40:00,875::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:40:00,876::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`78fe4911-5f73-4fe9-91e7-b226b808964a`::Granted request >Thread-338::DEBUG::2014-05-13 11:40:00,876::task::827::TaskManager.Task::(resourceAcquired) Task=`bfff2103-106a-4cf0-983b-eaf8eff7ad83`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:40:00,876::task::990::TaskManager.Task::(_decref) Task=`bfff2103-106a-4cf0-983b-eaf8eff7ad83`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:40:00,877::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:40:00,877::task::1185::TaskManager.Task::(prepare) Task=`bfff2103-106a-4cf0-983b-eaf8eff7ad83`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:40:00,877::task::595::TaskManager.Task::(_updateState) Task=`bfff2103-106a-4cf0-983b-eaf8eff7ad83`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:40:00,877::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:40:00,878::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:40:00,878::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:40:00,878::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:40:00,878::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:40:00,878::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:40:00,879::task::990::TaskManager.Task::(_decref) Task=`bfff2103-106a-4cf0-983b-eaf8eff7ad83`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:40:00,927::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:40:01,006::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359705 s, 28.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:40:03,019::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:40:03,093::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307869 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:40:03,436::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:40:04,900::task::595::TaskManager.Task::(_updateState) Task=`e4fbab6d-a7f6-4963-8ec6-0cc728c2d424`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:40:04,900::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:40:04,900::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000678332', 'lastCheck': '5.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:40:04,901::task::1185::TaskManager.Task::(prepare) Task=`e4fbab6d-a7f6-4963-8ec6-0cc728c2d424`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000678332', 'lastCheck': '5.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:40:04,901::task::595::TaskManager.Task::(_updateState) Task=`e4fbab6d-a7f6-4963-8ec6-0cc728c2d424`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:40:04,901::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:40:04,901::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:40:04,901::task::990::TaskManager.Task::(_decref) Task=`e4fbab6d-a7f6-4963-8ec6-0cc728c2d424`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:40:05,106::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:40:05,474::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.324934 s, 3.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:40:07,487::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:40:07,561::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310856 s, 32.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:40:09,171::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:40:09,180::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00216193 s, 1.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:40:09,574::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:40:09,652::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0350653 s, 29.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:40:10,897::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3c71a9dd] >Thread-13::DEBUG::2014-05-13 11:40:10,897::task::595::TaskManager.Task::(_updateState) Task=`f2d149fc-133b-461b-9f83-b6b06087fe67`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:40:10,898::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:40:10,898::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:40:10,898::task::1185::TaskManager.Task::(prepare) Task=`f2d149fc-133b-461b-9f83-b6b06087fe67`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:40:10,898::task::595::TaskManager.Task::(_updateState) Task=`f2d149fc-133b-461b-9f83-b6b06087fe67`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:40:10,899::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:40:10,899::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:40:10,899::task::990::TaskManager.Task::(_decref) Task=`f2d149fc-133b-461b-9f83-b6b06087fe67`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:40:10,903::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3c71a9dd] >Thread-338::DEBUG::2014-05-13 11:40:10,904::task::595::TaskManager.Task::(_updateState) Task=`7f6e82a3-1102-453b-a2d3-5e5b40d09212`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:40:10,904::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:40:10,904::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5d2b374c-d429-44be-ad4d-625d01406453`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:40:10,905::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:40:10,905::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:40:10,905::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5d2b374c-d429-44be-ad4d-625d01406453`::Granted request >Thread-338::DEBUG::2014-05-13 11:40:10,905::task::827::TaskManager.Task::(resourceAcquired) Task=`7f6e82a3-1102-453b-a2d3-5e5b40d09212`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:40:10,906::task::990::TaskManager.Task::(_decref) Task=`7f6e82a3-1102-453b-a2d3-5e5b40d09212`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:40:10,906::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:40:10,906::task::1185::TaskManager.Task::(prepare) Task=`7f6e82a3-1102-453b-a2d3-5e5b40d09212`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:40:10,907::task::595::TaskManager.Task::(_updateState) Task=`7f6e82a3-1102-453b-a2d3-5e5b40d09212`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:40:10,907::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:40:10,907::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:40:10,907::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:40:10,907::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:40:10,908::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:40:10,908::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:40:10,908::task::990::TaskManager.Task::(_decref) Task=`7f6e82a3-1102-453b-a2d3-5e5b40d09212`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:40:11,522::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:40:11,665::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:40:11,745::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0366771 s, 27.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:40:13,758::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:40:13,831::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309882 s, 33.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:40:14,543::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:40:15,844::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:40:15,924::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0367614 s, 27.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:40:17,564::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:40:17,937::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:40:18,025::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0453594 s, 22.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:40:19,188::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:40:19,196::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000795164 s, 5.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:40:20,038::task::595::TaskManager.Task::(_updateState) Task=`8ae67891-439a-42c9-9bcb-f82a96492249`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:40:20,039::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:40:20,040::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000795164', 'lastCheck': '0.8', 'valid': True}} >Dummy-337::DEBUG::2014-05-13 11:40:20,040::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 11:40:20,040::task::1185::TaskManager.Task::(prepare) Task=`8ae67891-439a-42c9-9bcb-f82a96492249`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000795164', 'lastCheck': '0.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:40:20,047::task::595::TaskManager.Task::(_updateState) Task=`8ae67891-439a-42c9-9bcb-f82a96492249`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:40:20,049::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:40:20,049::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:40:20,049::task::990::TaskManager.Task::(_decref) Task=`8ae67891-439a-42c9-9bcb-f82a96492249`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:40:20,124::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0410931 s, 24.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:40:20,584::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:40:20,926::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6602059] >Thread-13::DEBUG::2014-05-13 11:40:20,926::task::595::TaskManager.Task::(_updateState) Task=`c00e0f34-86b3-44d8-96e1-adf87f96a93a`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:40:20,926::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:40:20,927::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:40:20,927::task::1185::TaskManager.Task::(prepare) Task=`c00e0f34-86b3-44d8-96e1-adf87f96a93a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:40:20,927::task::595::TaskManager.Task::(_updateState) Task=`c00e0f34-86b3-44d8-96e1-adf87f96a93a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:40:20,927::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:40:20,927::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:40:20,927::task::990::TaskManager.Task::(_decref) Task=`c00e0f34-86b3-44d8-96e1-adf87f96a93a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:40:20,932::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6602059] >Thread-338::DEBUG::2014-05-13 11:40:20,932::task::595::TaskManager.Task::(_updateState) Task=`e13c762a-26da-49e1-a4dc-00547d177461`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:40:20,932::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:40:20,932::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c94a7d14-6ac8-483a-8504-b6242b5655fe`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:40:20,933::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:40:20,933::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:40:20,933::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c94a7d14-6ac8-483a-8504-b6242b5655fe`::Granted request >Thread-338::DEBUG::2014-05-13 11:40:20,933::task::827::TaskManager.Task::(resourceAcquired) Task=`e13c762a-26da-49e1-a4dc-00547d177461`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:40:20,933::task::990::TaskManager.Task::(_decref) Task=`e13c762a-26da-49e1-a4dc-00547d177461`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:40:20,934::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:40:20,934::task::1185::TaskManager.Task::(prepare) Task=`e13c762a-26da-49e1-a4dc-00547d177461`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:40:20,934::task::595::TaskManager.Task::(_updateState) Task=`e13c762a-26da-49e1-a4dc-00547d177461`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:40:20,935::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:40:20,935::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:40:20,935::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:40:20,935::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:40:20,935::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:40:20,935::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:40:20,936::task::990::TaskManager.Task::(_decref) Task=`e13c762a-26da-49e1-a4dc-00547d177461`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:40:22,137::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:40:22,212::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310716 s, 33.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:40:23,603::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:40:24,224::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:40:24,299::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316306 s, 32.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:40:26,312::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:40:26,395::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0409602 s, 25.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:40:26,622::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:40:28,409::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:40:28,484::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322541 s, 31.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:40:29,227::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:40:29,235::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000661338 s, 6.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:40:29,642::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:40:30,499::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:40:30,587::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0443421 s, 23.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:40:30,958::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:40:30,959::task::595::TaskManager.Task::(_updateState) Task=`567be569-2b4b-4ba2-a0fe-7feb9e6f30a1`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:40:30,959::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:40:30,960::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:40:30,960::task::1185::TaskManager.Task::(prepare) Task=`567be569-2b4b-4ba2-a0fe-7feb9e6f30a1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:40:30,960::task::595::TaskManager.Task::(_updateState) Task=`567be569-2b4b-4ba2-a0fe-7feb9e6f30a1`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:40:30,960::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:40:30,960::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:40:30,960::task::990::TaskManager.Task::(_decref) Task=`567be569-2b4b-4ba2-a0fe-7feb9e6f30a1`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:40:30,966::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:40:30,966::task::595::TaskManager.Task::(_updateState) Task=`13a1d78d-3c97-48e7-9094-94ee110d9634`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:40:30,967::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:40:30,967::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`14bc31f0-1b7a-4079-957b-b458367d6ad1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:40:30,967::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:40:30,968::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:40:30,968::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`14bc31f0-1b7a-4079-957b-b458367d6ad1`::Granted request >Thread-338::DEBUG::2014-05-13 11:40:30,968::task::827::TaskManager.Task::(resourceAcquired) Task=`13a1d78d-3c97-48e7-9094-94ee110d9634`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:40:30,968::task::990::TaskManager.Task::(_decref) Task=`13a1d78d-3c97-48e7-9094-94ee110d9634`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:40:30,969::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:40:30,969::task::1185::TaskManager.Task::(prepare) Task=`13a1d78d-3c97-48e7-9094-94ee110d9634`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:40:30,969::task::595::TaskManager.Task::(_updateState) Task=`13a1d78d-3c97-48e7-9094-94ee110d9634`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:40:30,969::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:40:30,970::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:40:30,970::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:40:30,970::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:40:30,970::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:40:30,970::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:40:30,971::task::990::TaskManager.Task::(_decref) Task=`13a1d78d-3c97-48e7-9094-94ee110d9634`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:40:32,600::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:40:32,667::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:40:32,699::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0543089 s, 18.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:40:34,712::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:40:34,788::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318694 s, 32.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:40:35,184::task::595::TaskManager.Task::(_updateState) Task=`040caff4-b907-4262-a624-0edf20786966`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:40:35,184::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:40:35,184::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000661338', 'lastCheck': '5.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:40:35,185::task::1185::TaskManager.Task::(prepare) Task=`040caff4-b907-4262-a624-0edf20786966`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000661338', 'lastCheck': '5.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:40:35,186::task::595::TaskManager.Task::(_updateState) Task=`040caff4-b907-4262-a624-0edf20786966`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:40:35,186::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:40:35,186::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:40:35,186::task::990::TaskManager.Task::(_decref) Task=`040caff4-b907-4262-a624-0edf20786966`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:40:35,698::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:40:36,800::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:40:36,877::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0324508 s, 31.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:40:38,722::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:40:38,890::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:40:38,968::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0347169 s, 29.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:40:39,243::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:40:39,250::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000607475 s, 6.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:40:40,981::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 11:40:40,992::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:40:40,993::task::595::TaskManager.Task::(_updateState) Task=`a5cff9e7-6fec-4d79-954a-3c0635427fd9`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:40:40,993::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:40:40,994::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:40:40,994::task::1185::TaskManager.Task::(prepare) Task=`a5cff9e7-6fec-4d79-954a-3c0635427fd9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:40:40,994::task::595::TaskManager.Task::(_updateState) Task=`a5cff9e7-6fec-4d79-954a-3c0635427fd9`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:40:40,994::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:40:40,994::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:40:40,995::task::990::TaskManager.Task::(_decref) Task=`a5cff9e7-6fec-4d79-954a-3c0635427fd9`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:40:41,002::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:40:41,004::task::595::TaskManager.Task::(_updateState) Task=`ba5eed44-fddd-49dc-9276-7a4719dab951`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:40:41,005::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:40:41,007::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`173332e4-45c9-447b-adc7-08d11c9cd745`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:40:41,008::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:40:41,009::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:40:41,010::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`173332e4-45c9-447b-adc7-08d11c9cd745`::Granted request >Thread-338::DEBUG::2014-05-13 11:40:41,012::task::827::TaskManager.Task::(resourceAcquired) Task=`ba5eed44-fddd-49dc-9276-7a4719dab951`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:40:41,013::task::990::TaskManager.Task::(_decref) Task=`ba5eed44-fddd-49dc-9276-7a4719dab951`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:40:41,017::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:40:41,018::task::1185::TaskManager.Task::(prepare) Task=`ba5eed44-fddd-49dc-9276-7a4719dab951`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:40:41,019::task::595::TaskManager.Task::(_updateState) Task=`ba5eed44-fddd-49dc-9276-7a4719dab951`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:40:41,020::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:40:41,021::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:40:41,022::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:40:41,024::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:40:41,024::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:40:41,025::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:40:41,027::task::990::TaskManager.Task::(_decref) Task=`ba5eed44-fddd-49dc-9276-7a4719dab951`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:40:41,087::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0644263 s, 15.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:40:41,754::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:40:43,100::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:40:43,173::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307045 s, 33.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:40:44,774::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:40:45,186::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:40:45,555::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.326168 s, 3.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:40:47,572::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:40:47,651::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362347 s, 28.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:40:47,803::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:40:49,257::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:40:49,265::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000737854 s, 5.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:40:49,664::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:40:49,743::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364973 s, 28.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:40:50,324::task::595::TaskManager.Task::(_updateState) Task=`f4473927-5487-4364-9245-4b43f46c1a7b`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:40:50,324::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:40:50,324::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000737854', 'lastCheck': '1.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:40:50,324::task::1185::TaskManager.Task::(prepare) Task=`f4473927-5487-4364-9245-4b43f46c1a7b`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000737854', 'lastCheck': '1.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:40:50,325::task::595::TaskManager.Task::(_updateState) Task=`f4473927-5487-4364-9245-4b43f46c1a7b`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:40:50,325::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:40:50,325::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:40:50,325::task::990::TaskManager.Task::(_decref) Task=`f4473927-5487-4364-9245-4b43f46c1a7b`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:40:50,821::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:40:51,048::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:40:51,048::task::595::TaskManager.Task::(_updateState) Task=`7bbdb613-47f8-4ed3-abbb-842705d6ee0b`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:40:51,049::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:40:51,049::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:40:51,049::task::1185::TaskManager.Task::(prepare) Task=`7bbdb613-47f8-4ed3-abbb-842705d6ee0b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:40:51,049::task::595::TaskManager.Task::(_updateState) Task=`7bbdb613-47f8-4ed3-abbb-842705d6ee0b`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:40:51,049::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:40:51,050::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:40:51,050::task::990::TaskManager.Task::(_decref) Task=`7bbdb613-47f8-4ed3-abbb-842705d6ee0b`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:40:51,055::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:40:51,055::task::595::TaskManager.Task::(_updateState) Task=`482188a7-916f-42fa-9bcd-6dbc057ba5c5`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:40:51,055::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:40:51,056::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`92a482df-842c-415e-8bb9-493ac81c7e46`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:40:51,056::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:40:51,056::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:40:51,056::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`92a482df-842c-415e-8bb9-493ac81c7e46`::Granted request >Thread-338::DEBUG::2014-05-13 11:40:51,057::task::827::TaskManager.Task::(resourceAcquired) Task=`482188a7-916f-42fa-9bcd-6dbc057ba5c5`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:40:51,057::task::990::TaskManager.Task::(_decref) Task=`482188a7-916f-42fa-9bcd-6dbc057ba5c5`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:40:51,057::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:40:51,057::task::1185::TaskManager.Task::(prepare) Task=`482188a7-916f-42fa-9bcd-6dbc057ba5c5`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:40:51,058::task::595::TaskManager.Task::(_updateState) Task=`482188a7-916f-42fa-9bcd-6dbc057ba5c5`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:40:51,058::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:40:51,058::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:40:51,058::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:40:51,058::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:40:51,058::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:40:51,059::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:40:51,059::task::990::TaskManager.Task::(_decref) Task=`482188a7-916f-42fa-9bcd-6dbc057ba5c5`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:40:51,756::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:40:51,830::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306907 s, 33.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:40:53,838::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:40:53,843::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:40:53,917::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318832 s, 32.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:40:55,930::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:40:56,010::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364911 s, 28.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:40:56,858::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:40:58,023::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:40:58,098::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323489 s, 31.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:40:59,272::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:40:59,280::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000587283 s, 7.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:40:59,881::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:41:00,112::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:00,201::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0468369 s, 21.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:41:01,078::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1ea398d3] >Thread-13::DEBUG::2014-05-13 11:41:01,079::task::595::TaskManager.Task::(_updateState) Task=`58f4bf2b-6619-4976-af64-3e703fcbadbc`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:41:01,079::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:41:01,079::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:41:01,080::task::1185::TaskManager.Task::(prepare) Task=`58f4bf2b-6619-4976-af64-3e703fcbadbc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:41:01,080::task::595::TaskManager.Task::(_updateState) Task=`58f4bf2b-6619-4976-af64-3e703fcbadbc`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:41:01,080::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:41:01,080::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:41:01,080::task::990::TaskManager.Task::(_decref) Task=`58f4bf2b-6619-4976-af64-3e703fcbadbc`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:41:01,085::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1ea398d3] >Thread-338::DEBUG::2014-05-13 11:41:01,085::task::595::TaskManager.Task::(_updateState) Task=`e5b701d4-3e1c-45d5-9503-7309ee64201c`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:41:01,085::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:41:01,086::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`003d567c-d06a-4eef-82c3-4e19d89f7766`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:41:01,086::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:41:01,086::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:41:01,086::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`003d567c-d06a-4eef-82c3-4e19d89f7766`::Granted request >Thread-338::DEBUG::2014-05-13 11:41:01,087::task::827::TaskManager.Task::(resourceAcquired) Task=`e5b701d4-3e1c-45d5-9503-7309ee64201c`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:41:01,087::task::990::TaskManager.Task::(_decref) Task=`e5b701d4-3e1c-45d5-9503-7309ee64201c`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:41:01,087::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:41:01,088::task::1185::TaskManager.Task::(prepare) Task=`e5b701d4-3e1c-45d5-9503-7309ee64201c`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:41:01,088::task::595::TaskManager.Task::(_updateState) Task=`e5b701d4-3e1c-45d5-9503-7309ee64201c`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:41:01,088::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:41:01,088::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:41:01,088::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:41:01,089::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:41:01,089::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:41:01,089::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:41:01,089::task::990::TaskManager.Task::(_decref) Task=`e5b701d4-3e1c-45d5-9503-7309ee64201c`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:41:02,215::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:02,295::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373346 s, 27.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:41:02,901::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:41:04,308::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:04,382::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311022 s, 32.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:41:05,463::task::595::TaskManager.Task::(_updateState) Task=`144f8933-918a-4d95-8592-b985703765ce`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:41:05,464::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:41:05,464::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000587283', 'lastCheck': '6.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:41:05,464::task::1185::TaskManager.Task::(prepare) Task=`144f8933-918a-4d95-8592-b985703765ce`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000587283', 'lastCheck': '6.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:41:05,464::task::595::TaskManager.Task::(_updateState) Task=`144f8933-918a-4d95-8592-b985703765ce`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:41:05,464::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:41:05,465::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:41:05,465::task::990::TaskManager.Task::(_decref) Task=`144f8933-918a-4d95-8592-b985703765ce`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:41:05,919::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:41:06,396::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:06,471::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0308217 s, 33.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:41:08,484::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:08,569::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0375277 s, 27.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:41:08,937::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:41:09,287::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:41:09,295::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000647439 s, 6.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:41:10,582::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:10,661::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379254 s, 27.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:41:11,113::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:41:11,114::task::595::TaskManager.Task::(_updateState) Task=`6ad1b5a2-31d1-4e49-b0f7-8bafa028b9e6`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:41:11,114::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:41:11,114::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:41:11,115::task::1185::TaskManager.Task::(prepare) Task=`6ad1b5a2-31d1-4e49-b0f7-8bafa028b9e6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:41:11,115::task::595::TaskManager.Task::(_updateState) Task=`6ad1b5a2-31d1-4e49-b0f7-8bafa028b9e6`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:41:11,115::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:41:11,115::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:41:11,115::task::990::TaskManager.Task::(_decref) Task=`6ad1b5a2-31d1-4e49-b0f7-8bafa028b9e6`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:41:11,120::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:41:11,120::task::595::TaskManager.Task::(_updateState) Task=`40ea1316-6b4f-42f2-940c-acf07511f69b`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:41:11,120::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:41:11,121::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b078e911-3c98-47c9-9e60-ed89d413b9b4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:41:11,121::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:41:11,121::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:41:11,121::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b078e911-3c98-47c9-9e60-ed89d413b9b4`::Granted request >Thread-338::DEBUG::2014-05-13 11:41:11,122::task::827::TaskManager.Task::(resourceAcquired) Task=`40ea1316-6b4f-42f2-940c-acf07511f69b`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:41:11,122::task::990::TaskManager.Task::(_decref) Task=`40ea1316-6b4f-42f2-940c-acf07511f69b`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:41:11,123::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:41:11,123::task::1185::TaskManager.Task::(prepare) Task=`40ea1316-6b4f-42f2-940c-acf07511f69b`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:41:11,123::task::595::TaskManager.Task::(_updateState) Task=`40ea1316-6b4f-42f2-940c-acf07511f69b`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:41:11,123::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:41:11,123::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:41:11,124::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:41:11,124::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:41:11,124::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:41:11,124::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:41:11,124::task::990::TaskManager.Task::(_decref) Task=`40ea1316-6b4f-42f2-940c-acf07511f69b`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:41:11,960::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:41:12,679::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:12,770::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0330848 s, 31.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:41:14,783::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:14,872::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0488827 s, 20.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:41:14,978::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:41:16,880::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:16,958::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363718 s, 28.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:41:17,999::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:41:18,971::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:19,046::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319932 s, 32.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:41:19,302::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:41:19,310::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000753345 s, 5.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:41:20,613::task::595::TaskManager.Task::(_updateState) Task=`9d30d57f-5eb9-4576-993f-b48fcfef8884`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:41:20,613::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:41:20,613::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000753345', 'lastCheck': '1.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:41:20,613::task::1185::TaskManager.Task::(prepare) Task=`9d30d57f-5eb9-4576-993f-b48fcfef8884`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000753345', 'lastCheck': '1.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:41:20,613::task::595::TaskManager.Task::(_updateState) Task=`9d30d57f-5eb9-4576-993f-b48fcfef8884`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:41:20,614::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:41:20,614::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:41:20,614::task::990::TaskManager.Task::(_decref) Task=`9d30d57f-5eb9-4576-993f-b48fcfef8884`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:41:21,018::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:41:21,059::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:21,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031078 s, 32.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:41:21,146::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [30b3ccf] >Thread-13::DEBUG::2014-05-13 11:41:21,146::task::595::TaskManager.Task::(_updateState) Task=`69740587-3fb1-44be-9f9c-b5ba418e5786`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:41:21,146::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:41:21,147::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:41:21,147::task::1185::TaskManager.Task::(prepare) Task=`69740587-3fb1-44be-9f9c-b5ba418e5786`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:41:21,147::task::595::TaskManager.Task::(_updateState) Task=`69740587-3fb1-44be-9f9c-b5ba418e5786`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:41:21,147::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:41:21,147::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:41:21,148::task::990::TaskManager.Task::(_decref) Task=`69740587-3fb1-44be-9f9c-b5ba418e5786`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:41:21,152::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [30b3ccf] >Thread-338::DEBUG::2014-05-13 11:41:21,153::task::595::TaskManager.Task::(_updateState) Task=`6429276d-aa1f-4c44-90f7-bcb033d3f9d5`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:41:21,153::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:41:21,153::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8bf36a10-9e53-4be8-bc39-b5a70a6fb304`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:41:21,154::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:41:21,154::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:41:21,154::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8bf36a10-9e53-4be8-bc39-b5a70a6fb304`::Granted request >Thread-338::DEBUG::2014-05-13 11:41:21,154::task::827::TaskManager.Task::(resourceAcquired) Task=`6429276d-aa1f-4c44-90f7-bcb033d3f9d5`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:41:21,155::task::990::TaskManager.Task::(_decref) Task=`6429276d-aa1f-4c44-90f7-bcb033d3f9d5`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:41:21,155::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:41:21,155::task::1185::TaskManager.Task::(prepare) Task=`6429276d-aa1f-4c44-90f7-bcb033d3f9d5`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:41:21,156::task::595::TaskManager.Task::(_updateState) Task=`6429276d-aa1f-4c44-90f7-bcb033d3f9d5`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:41:21,156::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:41:21,156::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:41:21,156::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:41:21,156::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:41:21,157::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:41:21,157::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:41:21,157::task::990::TaskManager.Task::(_decref) Task=`6429276d-aa1f-4c44-90f7-bcb033d3f9d5`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:41:23,147::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:23,228::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0368171 s, 27.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:41:24,036::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:41:25,241::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:25,318::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318252 s, 32.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:41:27,056::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:41:27,331::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:27,742::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.366598 s, 2.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:41:29,317::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:41:29,325::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000739014 s, 5.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:41:29,756::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:29,837::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.039048 s, 26.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:41:30,076::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:41:31,176::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:41:31,176::task::595::TaskManager.Task::(_updateState) Task=`444a14ce-8008-41a3-9687-c03cd4d2b110`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:41:31,176::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:41:31,177::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:41:31,177::task::1185::TaskManager.Task::(prepare) Task=`444a14ce-8008-41a3-9687-c03cd4d2b110`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:41:31,177::task::595::TaskManager.Task::(_updateState) Task=`444a14ce-8008-41a3-9687-c03cd4d2b110`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:41:31,177::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:41:31,177::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:41:31,178::task::990::TaskManager.Task::(_decref) Task=`444a14ce-8008-41a3-9687-c03cd4d2b110`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:41:31,182::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:41:31,182::task::595::TaskManager.Task::(_updateState) Task=`5fd354da-e518-4a39-8f07-046537765795`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:41:31,182::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:41:31,183::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9e2481a2-0fb2-439a-b7b2-e7dcc35ce3b6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:41:31,183::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:41:31,183::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:41:31,184::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9e2481a2-0fb2-439a-b7b2-e7dcc35ce3b6`::Granted request >Thread-338::DEBUG::2014-05-13 11:41:31,184::task::827::TaskManager.Task::(resourceAcquired) Task=`5fd354da-e518-4a39-8f07-046537765795`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:41:31,184::task::990::TaskManager.Task::(_decref) Task=`5fd354da-e518-4a39-8f07-046537765795`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:41:31,185::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:41:31,185::task::1185::TaskManager.Task::(prepare) Task=`5fd354da-e518-4a39-8f07-046537765795`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:41:31,185::task::595::TaskManager.Task::(_updateState) Task=`5fd354da-e518-4a39-8f07-046537765795`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:41:31,185::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:41:31,186::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:41:31,186::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:41:31,186::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:41:31,186::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:41:31,186::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:41:31,187::task::990::TaskManager.Task::(_decref) Task=`5fd354da-e518-4a39-8f07-046537765795`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:41:31,850::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:31,931::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387461 s, 26.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:41:33,094::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:41:33,944::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:34,018::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313312 s, 32.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:41:35,752::task::595::TaskManager.Task::(_updateState) Task=`b0f5668d-bfba-4acc-8879-995ba3006d8d`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:41:35,752::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:41:35,752::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000739014', 'lastCheck': '6.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:41:35,753::task::1185::TaskManager.Task::(prepare) Task=`b0f5668d-bfba-4acc-8879-995ba3006d8d`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000739014', 'lastCheck': '6.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:41:35,753::task::595::TaskManager.Task::(_updateState) Task=`b0f5668d-bfba-4acc-8879-995ba3006d8d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:41:35,753::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:41:35,753::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:41:35,753::task::990::TaskManager.Task::(_decref) Task=`b0f5668d-bfba-4acc-8879-995ba3006d8d`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:41:36,031::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:36,107::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318025 s, 32.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:41:36,112::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:41:38,123::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:38,202::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360235 s, 28.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:41:39,131::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:41:39,332::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:41:39,343::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00377531 s, 1.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:41:40,215::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:40,308::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0491537 s, 20.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:41:41,212::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4478bd69] >Thread-13::DEBUG::2014-05-13 11:41:41,212::task::595::TaskManager.Task::(_updateState) Task=`02c5d207-a531-4306-82c5-f7de80788c92`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:41:41,213::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:41:41,213::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:41:41,213::task::1185::TaskManager.Task::(prepare) Task=`02c5d207-a531-4306-82c5-f7de80788c92`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:41:41,213::task::595::TaskManager.Task::(_updateState) Task=`02c5d207-a531-4306-82c5-f7de80788c92`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:41:41,214::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:41:41,214::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:41:41,214::task::990::TaskManager.Task::(_decref) Task=`02c5d207-a531-4306-82c5-f7de80788c92`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:41:41,218::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4478bd69] >Thread-338::DEBUG::2014-05-13 11:41:41,219::task::595::TaskManager.Task::(_updateState) Task=`e5b46255-5027-463d-8529-cc9a158732ad`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:41:41,219::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:41:41,219::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6486a890-f8bb-4176-8715-e4c47be89332`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:41:41,219::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:41:41,220::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:41:41,220::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6486a890-f8bb-4176-8715-e4c47be89332`::Granted request >Thread-338::DEBUG::2014-05-13 11:41:41,220::task::827::TaskManager.Task::(resourceAcquired) Task=`e5b46255-5027-463d-8529-cc9a158732ad`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:41:41,220::task::990::TaskManager.Task::(_decref) Task=`e5b46255-5027-463d-8529-cc9a158732ad`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:41:41,221::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:41:41,221::task::1185::TaskManager.Task::(prepare) Task=`e5b46255-5027-463d-8529-cc9a158732ad`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:41:41,222::task::595::TaskManager.Task::(_updateState) Task=`e5b46255-5027-463d-8529-cc9a158732ad`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:41:41,222::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:41:41,222::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:41:41,222::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:41:41,222::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:41:41,222::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:41:41,223::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:41:41,223::task::990::TaskManager.Task::(_decref) Task=`e5b46255-5027-463d-8529-cc9a158732ad`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:41:42,149::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:41:42,321::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:42,395::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314837 s, 32.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:41:44,408::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:44,489::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382619 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:41:45,167::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:41:46,502::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:46,582::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382388 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:41:48,185::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:41:48,595::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:48,671::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0334059 s, 30.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:41:49,350::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:41:49,358::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000760206 s, 5.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:41:50,684::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:50,764::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382938 s, 26.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:41:50,897::task::595::TaskManager.Task::(_updateState) Task=`6d80ac3d-6304-4e21-a617-f882ead66e0d`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:41:50,898::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:41:50,898::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000760206', 'lastCheck': '1.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:41:50,898::task::1185::TaskManager.Task::(prepare) Task=`6d80ac3d-6304-4e21-a617-f882ead66e0d`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000760206', 'lastCheck': '1.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:41:50,898::task::595::TaskManager.Task::(_updateState) Task=`6d80ac3d-6304-4e21-a617-f882ead66e0d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:41:50,898::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:41:50,899::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:41:50,899::task::990::TaskManager.Task::(_decref) Task=`6d80ac3d-6304-4e21-a617-f882ead66e0d`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:41:51,203::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:41:51,242::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:41:51,243::task::595::TaskManager.Task::(_updateState) Task=`c100e663-a047-45cf-a42e-1e0101067230`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:41:51,243::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:41:51,243::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:41:51,243::task::1185::TaskManager.Task::(prepare) Task=`c100e663-a047-45cf-a42e-1e0101067230`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:41:51,244::task::595::TaskManager.Task::(_updateState) Task=`c100e663-a047-45cf-a42e-1e0101067230`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:41:51,244::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:41:51,244::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:41:51,244::task::990::TaskManager.Task::(_decref) Task=`c100e663-a047-45cf-a42e-1e0101067230`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:41:51,249::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:41:51,249::task::595::TaskManager.Task::(_updateState) Task=`5d189ffa-ac43-4068-b872-c4c90eb1009f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:41:51,249::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:41:51,250::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`7199daea-f801-402e-9ac5-81a7d693fa9c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:41:51,251::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:41:51,251::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:41:51,251::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`7199daea-f801-402e-9ac5-81a7d693fa9c`::Granted request >Thread-338::DEBUG::2014-05-13 11:41:51,251::task::827::TaskManager.Task::(resourceAcquired) Task=`5d189ffa-ac43-4068-b872-c4c90eb1009f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:41:51,251::task::990::TaskManager.Task::(_decref) Task=`5d189ffa-ac43-4068-b872-c4c90eb1009f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:41:51,252::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:41:51,252::task::1185::TaskManager.Task::(prepare) Task=`5d189ffa-ac43-4068-b872-c4c90eb1009f`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:41:51,252::task::595::TaskManager.Task::(_updateState) Task=`5d189ffa-ac43-4068-b872-c4c90eb1009f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:41:51,253::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:41:51,253::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:41:51,253::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:41:51,253::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:41:51,253::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:41:51,253::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:41:51,254::task::990::TaskManager.Task::(_decref) Task=`5d189ffa-ac43-4068-b872-c4c90eb1009f`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:41:52,776::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:52,865::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319521 s, 32.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:41:54,222::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:41:54,878::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:55,251::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.330811 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:41:57,248::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:41:57,264::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:41:57,339::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309986 s, 33.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:41:59,352::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 11:41:59,366::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:41:59,382::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000600894 s, 6.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:41:59,467::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0738716 s, 13.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:42:00,271::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:42:01,272::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:42:01,272::task::595::TaskManager.Task::(_updateState) Task=`6a5a71a3-2dbf-403b-a2d6-48189111545c`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:42:01,272::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:42:01,273::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:42:01,273::task::1185::TaskManager.Task::(prepare) Task=`6a5a71a3-2dbf-403b-a2d6-48189111545c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:42:01,273::task::595::TaskManager.Task::(_updateState) Task=`6a5a71a3-2dbf-403b-a2d6-48189111545c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:42:01,274::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:42:01,274::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:42:01,274::task::990::TaskManager.Task::(_decref) Task=`6a5a71a3-2dbf-403b-a2d6-48189111545c`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:42:01,283::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:42:01,283::task::595::TaskManager.Task::(_updateState) Task=`90708b63-910b-4731-b6ff-2fadb71dd4ca`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:42:01,283::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:42:01,284::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3fa7c29b-222f-4352-84fd-f095d475ffef`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:42:01,284::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:42:01,284::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:42:01,284::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3fa7c29b-222f-4352-84fd-f095d475ffef`::Granted request >Thread-338::DEBUG::2014-05-13 11:42:01,285::task::827::TaskManager.Task::(resourceAcquired) Task=`90708b63-910b-4731-b6ff-2fadb71dd4ca`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:42:01,285::task::990::TaskManager.Task::(_decref) Task=`90708b63-910b-4731-b6ff-2fadb71dd4ca`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:42:01,286::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:42:01,286::task::1185::TaskManager.Task::(prepare) Task=`90708b63-910b-4731-b6ff-2fadb71dd4ca`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:42:01,286::task::595::TaskManager.Task::(_updateState) Task=`90708b63-910b-4731-b6ff-2fadb71dd4ca`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:42:01,286::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:42:01,286::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:42:01,287::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:42:01,287::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:42:01,287::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:42:01,287::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:42:01,287::task::990::TaskManager.Task::(_decref) Task=`90708b63-910b-4731-b6ff-2fadb71dd4ca`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:42:01,480::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:01,558::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355632 s, 28.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:42:03,293::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:42:03,571::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:03,645::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307457 s, 33.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:42:05,658::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:05,738::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0368997 s, 27.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:42:06,035::task::595::TaskManager.Task::(_updateState) Task=`081244c4-4d10-47be-b53e-6eb6ad22dd4e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:42:06,035::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:42:06,035::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000600894', 'lastCheck': '6.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:42:06,035::task::1185::TaskManager.Task::(prepare) Task=`081244c4-4d10-47be-b53e-6eb6ad22dd4e`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000600894', 'lastCheck': '6.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:42:06,035::task::595::TaskManager.Task::(_updateState) Task=`081244c4-4d10-47be-b53e-6eb6ad22dd4e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:42:06,036::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:42:06,036::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:42:06,036::task::990::TaskManager.Task::(_decref) Task=`081244c4-4d10-47be-b53e-6eb6ad22dd4e`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:42:06,316::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:42:07,751::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:07,838::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0444554 s, 23.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:42:09,335::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:42:09,410::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:42:09,417::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000682264 s, 6.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:42:09,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:09,930::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363975 s, 28.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:42:11,309::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:42:11,310::task::595::TaskManager.Task::(_updateState) Task=`1bce99c1-4f20-4fe3-be80-cb4b9dd23761`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:42:11,310::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:42:11,311::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:42:11,311::task::1185::TaskManager.Task::(prepare) Task=`1bce99c1-4f20-4fe3-be80-cb4b9dd23761`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:42:11,311::task::595::TaskManager.Task::(_updateState) Task=`1bce99c1-4f20-4fe3-be80-cb4b9dd23761`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:42:11,311::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:42:11,311::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:42:11,311::task::990::TaskManager.Task::(_decref) Task=`1bce99c1-4f20-4fe3-be80-cb4b9dd23761`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:42:11,317::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:42:11,317::task::595::TaskManager.Task::(_updateState) Task=`eabe2d25-b4ea-4841-b198-b0427cfa3857`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:42:11,317::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:42:11,318::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2bcdb816-7c0f-4d91-9e65-538f69da0d27`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:42:11,318::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:42:11,318::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:42:11,318::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2bcdb816-7c0f-4d91-9e65-538f69da0d27`::Granted request >Thread-338::DEBUG::2014-05-13 11:42:11,318::task::827::TaskManager.Task::(resourceAcquired) Task=`eabe2d25-b4ea-4841-b198-b0427cfa3857`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:42:11,319::task::990::TaskManager.Task::(_decref) Task=`eabe2d25-b4ea-4841-b198-b0427cfa3857`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:42:11,319::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:42:11,320::task::1185::TaskManager.Task::(prepare) Task=`eabe2d25-b4ea-4841-b198-b0427cfa3857`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:42:11,320::task::595::TaskManager.Task::(_updateState) Task=`eabe2d25-b4ea-4841-b198-b0427cfa3857`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:42:11,320::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:42:11,320::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:42:11,320::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:42:11,321::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:42:11,321::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:42:11,321::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:42:11,321::task::990::TaskManager.Task::(_decref) Task=`eabe2d25-b4ea-4841-b198-b0427cfa3857`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:42:11,943::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:12,019::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322484 s, 31.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:42:12,353::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:42:14,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:14,112::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0370905 s, 27.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:42:15,373::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:42:16,125::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:16,204::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379257 s, 27.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:42:18,217::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:18,291::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307982 s, 33.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:42:18,392::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:42:19,425::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:42:19,433::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000707212 s, 5.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:42:20,304::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:20,384::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.038105 s, 26.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:42:21,180::task::595::TaskManager.Task::(_updateState) Task=`8a0b66b1-b151-49cf-a9b6-6f7a95cdb8a1`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:42:21,181::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:42:21,181::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000707212', 'lastCheck': '1.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:42:21,181::task::1185::TaskManager.Task::(prepare) Task=`8a0b66b1-b151-49cf-a9b6-6f7a95cdb8a1`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000707212', 'lastCheck': '1.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:42:21,181::task::595::TaskManager.Task::(_updateState) Task=`8a0b66b1-b151-49cf-a9b6-6f7a95cdb8a1`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:42:21,181::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:42:21,182::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:42:21,182::task::990::TaskManager.Task::(_decref) Task=`8a0b66b1-b151-49cf-a9b6-6f7a95cdb8a1`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 11:42:21,340::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4a63f6c8] >Thread-13::DEBUG::2014-05-13 11:42:21,340::task::595::TaskManager.Task::(_updateState) Task=`40bc069b-6752-498e-a28f-87b665ed62c5`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:42:21,341::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:42:21,341::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:42:21,341::task::1185::TaskManager.Task::(prepare) Task=`40bc069b-6752-498e-a28f-87b665ed62c5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:42:21,341::task::595::TaskManager.Task::(_updateState) Task=`40bc069b-6752-498e-a28f-87b665ed62c5`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:42:21,341::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:42:21,342::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:42:21,342::task::990::TaskManager.Task::(_decref) Task=`40bc069b-6752-498e-a28f-87b665ed62c5`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:42:21,346::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4a63f6c8] >Thread-338::DEBUG::2014-05-13 11:42:21,346::task::595::TaskManager.Task::(_updateState) Task=`c77503e8-a25d-4bf3-9fe3-0fed2184157b`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:42:21,346::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:42:21,347::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d60ec9ac-fec1-4032-9254-dc3a05b8264e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:42:21,347::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:42:21,347::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:42:21,347::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d60ec9ac-fec1-4032-9254-dc3a05b8264e`::Granted request >Thread-338::DEBUG::2014-05-13 11:42:21,348::task::827::TaskManager.Task::(resourceAcquired) Task=`c77503e8-a25d-4bf3-9fe3-0fed2184157b`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:42:21,348::task::990::TaskManager.Task::(_decref) Task=`c77503e8-a25d-4bf3-9fe3-0fed2184157b`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:42:21,349::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:42:21,349::task::1185::TaskManager.Task::(prepare) Task=`c77503e8-a25d-4bf3-9fe3-0fed2184157b`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:42:21,349::task::595::TaskManager.Task::(_updateState) Task=`c77503e8-a25d-4bf3-9fe3-0fed2184157b`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:42:21,349::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:42:21,349::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:42:21,350::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:42:21,350::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:42:21,350::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:42:21,350::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:42:21,350::task::990::TaskManager.Task::(_decref) Task=`c77503e8-a25d-4bf3-9fe3-0fed2184157b`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:42:21,412::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:42:22,397::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:22,489::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0503637 s, 20.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:42:24,432::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:42:24,501::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:24,576::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0332989 s, 30.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:42:26,590::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:26,670::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381239 s, 26.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:42:27,461::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:42:28,683::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:28,758::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318198 s, 32.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:42:29,440::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:42:29,448::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000675539 s, 6.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:42:30,489::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:42:30,770::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:30,846::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0338704 s, 30.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:42:31,369::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:42:31,370::task::595::TaskManager.Task::(_updateState) Task=`f0ea2df6-3f2e-4e09-a99c-1aa29d1d9075`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:42:31,370::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:42:31,370::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:42:31,370::task::1185::TaskManager.Task::(prepare) Task=`f0ea2df6-3f2e-4e09-a99c-1aa29d1d9075`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:42:31,371::task::595::TaskManager.Task::(_updateState) Task=`f0ea2df6-3f2e-4e09-a99c-1aa29d1d9075`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:42:31,371::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:42:31,371::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:42:31,371::task::990::TaskManager.Task::(_decref) Task=`f0ea2df6-3f2e-4e09-a99c-1aa29d1d9075`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:42:31,376::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:42:31,376::task::595::TaskManager.Task::(_updateState) Task=`e83d116a-6e58-4f7e-9904-1eab5d23fc86`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:42:31,377::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:42:31,377::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`05948888-877a-4673-a245-a97cd3ccd5de`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:42:31,377::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:42:31,377::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:42:31,378::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`05948888-877a-4673-a245-a97cd3ccd5de`::Granted request >Thread-338::DEBUG::2014-05-13 11:42:31,378::task::827::TaskManager.Task::(resourceAcquired) Task=`e83d116a-6e58-4f7e-9904-1eab5d23fc86`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:42:31,378::task::990::TaskManager.Task::(_decref) Task=`e83d116a-6e58-4f7e-9904-1eab5d23fc86`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:42:31,379::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:42:31,379::task::1185::TaskManager.Task::(prepare) Task=`e83d116a-6e58-4f7e-9904-1eab5d23fc86`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:42:31,379::task::595::TaskManager.Task::(_updateState) Task=`e83d116a-6e58-4f7e-9904-1eab5d23fc86`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:42:31,379::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:42:31,380::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:42:31,380::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:42:31,380::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:42:31,380::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:42:31,380::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:42:31,381::task::990::TaskManager.Task::(_decref) Task=`e83d116a-6e58-4f7e-9904-1eab5d23fc86`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:42:32,860::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:32,941::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383513 s, 26.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:42:33,506::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:42:34,949::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:35,024::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319919 s, 32.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:42:36,319::task::595::TaskManager.Task::(_updateState) Task=`00da3b32-05ac-4282-94c8-3861c4defb62`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:42:36,319::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:42:36,320::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000675539', 'lastCheck': '6.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:42:36,320::task::1185::TaskManager.Task::(prepare) Task=`00da3b32-05ac-4282-94c8-3861c4defb62`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000675539', 'lastCheck': '6.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:42:36,320::task::595::TaskManager.Task::(_updateState) Task=`00da3b32-05ac-4282-94c8-3861c4defb62`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:42:36,320::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:42:36,320::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:42:36,321::task::990::TaskManager.Task::(_decref) Task=`00da3b32-05ac-4282-94c8-3861c4defb62`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:42:36,525::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:42:37,037::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:37,133::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0512072 s, 20.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:42:39,146::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:39,221::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0338348 s, 30.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:42:39,455::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:42:39,464::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000633132 s, 6.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:42:39,543::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:42:41,235::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:41,315::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0369081 s, 27.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:42:41,399::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1cb1b71a] >Thread-13::DEBUG::2014-05-13 11:42:41,400::task::595::TaskManager.Task::(_updateState) Task=`a5b980ee-e818-41c8-9099-92cdb038f2b8`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:42:41,400::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:42:41,400::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:42:41,401::task::1185::TaskManager.Task::(prepare) Task=`a5b980ee-e818-41c8-9099-92cdb038f2b8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:42:41,401::task::595::TaskManager.Task::(_updateState) Task=`a5b980ee-e818-41c8-9099-92cdb038f2b8`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:42:41,401::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:42:41,401::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:42:41,401::task::990::TaskManager.Task::(_decref) Task=`a5b980ee-e818-41c8-9099-92cdb038f2b8`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:42:41,695::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1cb1b71a] >Thread-338::DEBUG::2014-05-13 11:42:41,695::task::595::TaskManager.Task::(_updateState) Task=`3931bdf7-1d26-4c4a-b7a1-22d734ab794b`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:42:41,696::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:42:41,696::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`17ccc0b3-0c28-478a-b626-d64c113c06ff`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:42:41,696::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:42:41,696::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:42:41,697::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`17ccc0b3-0c28-478a-b626-d64c113c06ff`::Granted request >Thread-338::DEBUG::2014-05-13 11:42:41,697::task::827::TaskManager.Task::(resourceAcquired) Task=`3931bdf7-1d26-4c4a-b7a1-22d734ab794b`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:42:41,697::task::990::TaskManager.Task::(_decref) Task=`3931bdf7-1d26-4c4a-b7a1-22d734ab794b`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:42:41,698::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:42:41,698::task::1185::TaskManager.Task::(prepare) Task=`3931bdf7-1d26-4c4a-b7a1-22d734ab794b`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:42:41,698::task::595::TaskManager.Task::(_updateState) Task=`3931bdf7-1d26-4c4a-b7a1-22d734ab794b`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:42:41,698::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:42:41,699::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:42:41,699::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:42:41,699::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:42:41,699::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:42:41,699::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:42:41,700::task::990::TaskManager.Task::(_decref) Task=`3931bdf7-1d26-4c4a-b7a1-22d734ab794b`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:42:42,563::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:42:43,328::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:43,404::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.032239 s, 31.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:42:45,418::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:45,494::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322815 s, 31.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:42:45,582::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:42:47,507::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:47,588::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365848 s, 28.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:42:48,600::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:42:49,470::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:42:49,478::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000741888 s, 5.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:42:49,600::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:50,012::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.368457 s, 2.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:42:51,458::task::595::TaskManager.Task::(_updateState) Task=`e28821b6-fd6a-487c-bc57-b49425b0f2d9`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:42:51,458::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:42:51,459::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000741888', 'lastCheck': '2.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:42:51,459::task::1185::TaskManager.Task::(prepare) Task=`e28821b6-fd6a-487c-bc57-b49425b0f2d9`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000741888', 'lastCheck': '2.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:42:51,459::task::595::TaskManager.Task::(_updateState) Task=`e28821b6-fd6a-487c-bc57-b49425b0f2d9`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:42:51,459::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:42:51,459::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:42:51,459::task::990::TaskManager.Task::(_decref) Task=`e28821b6-fd6a-487c-bc57-b49425b0f2d9`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:42:51,620::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:42:51,723::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:42:51,723::task::595::TaskManager.Task::(_updateState) Task=`8538b2ea-4bb6-4256-9bda-65fb264e793a`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:42:51,723::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:42:51,724::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:42:51,724::task::1185::TaskManager.Task::(prepare) Task=`8538b2ea-4bb6-4256-9bda-65fb264e793a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:42:51,724::task::595::TaskManager.Task::(_updateState) Task=`8538b2ea-4bb6-4256-9bda-65fb264e793a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:42:51,724::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:42:51,724::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:42:51,725::task::990::TaskManager.Task::(_decref) Task=`8538b2ea-4bb6-4256-9bda-65fb264e793a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:42:51,730::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:42:51,730::task::595::TaskManager.Task::(_updateState) Task=`bb086e27-5a42-40ea-a79b-56022c6327aa`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:42:51,730::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:42:51,731::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4da0c87a-c929-47be-b0fc-e6a5b46dbeae`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:42:51,731::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:42:51,731::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:42:51,731::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4da0c87a-c929-47be-b0fc-e6a5b46dbeae`::Granted request >Thread-338::DEBUG::2014-05-13 11:42:51,732::task::827::TaskManager.Task::(resourceAcquired) Task=`bb086e27-5a42-40ea-a79b-56022c6327aa`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:42:51,732::task::990::TaskManager.Task::(_decref) Task=`bb086e27-5a42-40ea-a79b-56022c6327aa`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:42:51,732::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:42:51,733::task::1185::TaskManager.Task::(prepare) Task=`bb086e27-5a42-40ea-a79b-56022c6327aa`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:42:51,733::task::595::TaskManager.Task::(_updateState) Task=`bb086e27-5a42-40ea-a79b-56022c6327aa`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:42:51,733::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:42:51,733::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:42:51,733::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:42:51,734::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:42:51,734::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:42:51,734::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:42:51,734::task::990::TaskManager.Task::(_decref) Task=`bb086e27-5a42-40ea-a79b-56022c6327aa`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:42:52,025::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:52,104::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361195 s, 28.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:42:54,117::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:54,191::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322121 s, 31.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:42:54,640::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:42:56,204::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:56,283::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360971 s, 28.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:42:57,659::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:42:58,296::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:42:58,371::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319522 s, 32.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:42:59,485::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:42:59,493::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000611634 s, 6.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:43:00,384::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:00,459::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317582 s, 32.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:43:00,677::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:43:01,755::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:43:01,755::task::595::TaskManager.Task::(_updateState) Task=`a693a98b-7236-4199-a663-733814747f59`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:43:01,755::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:43:01,756::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:43:01,756::task::1185::TaskManager.Task::(prepare) Task=`a693a98b-7236-4199-a663-733814747f59`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:43:01,756::task::595::TaskManager.Task::(_updateState) Task=`a693a98b-7236-4199-a663-733814747f59`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:43:01,756::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:43:01,756::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:43:01,757::task::990::TaskManager.Task::(_decref) Task=`a693a98b-7236-4199-a663-733814747f59`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:43:01,761::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:43:01,762::task::595::TaskManager.Task::(_updateState) Task=`7e56b6c2-6148-479a-9323-8ef405d48480`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:43:01,762::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:43:01,762::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`be93c616-f010-43b5-9ec8-c04193b61bf4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:43:01,762::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:43:01,763::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:43:01,763::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`be93c616-f010-43b5-9ec8-c04193b61bf4`::Granted request >Thread-338::DEBUG::2014-05-13 11:43:01,764::task::827::TaskManager.Task::(resourceAcquired) Task=`7e56b6c2-6148-479a-9323-8ef405d48480`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:43:01,764::task::990::TaskManager.Task::(_decref) Task=`7e56b6c2-6148-479a-9323-8ef405d48480`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:43:01,765::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:43:01,765::task::1185::TaskManager.Task::(prepare) Task=`7e56b6c2-6148-479a-9323-8ef405d48480`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:43:01,765::task::595::TaskManager.Task::(_updateState) Task=`7e56b6c2-6148-479a-9323-8ef405d48480`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:43:01,766::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:43:01,766::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:43:01,766::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:43:01,766::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:43:01,766::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:43:01,767::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:43:01,767::task::990::TaskManager.Task::(_decref) Task=`7e56b6c2-6148-479a-9323-8ef405d48480`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:43:02,473::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:02,567::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0510326 s, 20.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:43:03,697::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:43:04,580::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:04,654::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306968 s, 33.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:43:06,586::task::595::TaskManager.Task::(_updateState) Task=`221d78ca-2e23-4949-a4a5-7a385f54ce68`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:43:06,587::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:43:06,587::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000611634', 'lastCheck': '7.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:43:06,587::task::1185::TaskManager.Task::(prepare) Task=`221d78ca-2e23-4949-a4a5-7a385f54ce68`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000611634', 'lastCheck': '7.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:43:06,587::task::595::TaskManager.Task::(_updateState) Task=`221d78ca-2e23-4949-a4a5-7a385f54ce68`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:43:06,587::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:43:06,588::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:43:06,588::task::990::TaskManager.Task::(_decref) Task=`221d78ca-2e23-4949-a4a5-7a385f54ce68`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:43:06,667::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:43:06,719::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:43:06,749::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036488 s, 28.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:43:08,763::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:08,842::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356704 s, 28.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:43:09,500::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:43:09,508::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000755636 s, 5.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:43:09,738::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:43:10,855::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:10,934::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352392 s, 29.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:43:11,785::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1c80df6] >Thread-13::DEBUG::2014-05-13 11:43:11,786::task::595::TaskManager.Task::(_updateState) Task=`1e603832-24db-462d-986c-ce32aa8d7e18`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:43:11,786::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:43:11,787::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:43:11,787::task::1185::TaskManager.Task::(prepare) Task=`1e603832-24db-462d-986c-ce32aa8d7e18`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:43:11,787::task::595::TaskManager.Task::(_updateState) Task=`1e603832-24db-462d-986c-ce32aa8d7e18`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:43:11,787::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:43:11,787::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:43:11,788::task::990::TaskManager.Task::(_decref) Task=`1e603832-24db-462d-986c-ce32aa8d7e18`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:43:11,792::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1c80df6] >Thread-338::DEBUG::2014-05-13 11:43:11,792::task::595::TaskManager.Task::(_updateState) Task=`362a1653-2d04-4fed-8022-bd13140807d9`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:43:11,793::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:43:11,793::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c9b76c51-898c-41ba-9261-d96eebaa5a04`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:43:11,793::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:43:11,793::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:43:11,794::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c9b76c51-898c-41ba-9261-d96eebaa5a04`::Granted request >Thread-338::DEBUG::2014-05-13 11:43:11,794::task::827::TaskManager.Task::(resourceAcquired) Task=`362a1653-2d04-4fed-8022-bd13140807d9`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:43:11,794::task::990::TaskManager.Task::(_decref) Task=`362a1653-2d04-4fed-8022-bd13140807d9`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:43:11,795::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:43:11,795::task::1185::TaskManager.Task::(prepare) Task=`362a1653-2d04-4fed-8022-bd13140807d9`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:43:11,795::task::595::TaskManager.Task::(_updateState) Task=`362a1653-2d04-4fed-8022-bd13140807d9`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:43:11,795::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:43:11,796::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:43:11,796::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:43:11,796::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:43:11,796::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:43:11,796::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:43:11,797::task::990::TaskManager.Task::(_decref) Task=`362a1653-2d04-4fed-8022-bd13140807d9`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:43:12,758::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:43:12,946::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:13,022::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318626 s, 32.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:43:15,035::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:15,117::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387332 s, 26.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:43:15,781::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:43:17,130::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:17,221::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0490597 s, 20.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:43:18,798::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:43:19,234::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:19,309::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316436 s, 32.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:43:19,515::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:43:19,523::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000692712 s, 5.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:43:21,322::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:21,406::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0386979 s, 26.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:43:21,724::task::595::TaskManager.Task::(_updateState) Task=`1a53dae0-a734-4a44-aff9-6f65a2dd2258`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:43:21,724::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:43:21,725::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000692712', 'lastCheck': '2.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:43:21,725::task::1185::TaskManager.Task::(prepare) Task=`1a53dae0-a734-4a44-aff9-6f65a2dd2258`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000692712', 'lastCheck': '2.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:43:21,725::task::595::TaskManager.Task::(_updateState) Task=`1a53dae0-a734-4a44-aff9-6f65a2dd2258`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:43:21,725::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:43:21,725::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:43:21,725::task::990::TaskManager.Task::(_decref) Task=`1a53dae0-a734-4a44-aff9-6f65a2dd2258`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:43:21,818::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:43:21,819::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:43:21,819::task::595::TaskManager.Task::(_updateState) Task=`ce9b23e3-f913-41d8-bacd-2584ebdc41c6`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:43:21,819::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:43:21,820::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:43:21,820::task::1185::TaskManager.Task::(prepare) Task=`ce9b23e3-f913-41d8-bacd-2584ebdc41c6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:43:21,820::task::595::TaskManager.Task::(_updateState) Task=`ce9b23e3-f913-41d8-bacd-2584ebdc41c6`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:43:21,820::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:43:21,820::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:43:21,820::task::990::TaskManager.Task::(_decref) Task=`ce9b23e3-f913-41d8-bacd-2584ebdc41c6`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:43:21,866::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:43:21,866::task::595::TaskManager.Task::(_updateState) Task=`ac612fd5-ccf9-4b11-8672-51936ad11278`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:43:21,867::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:43:21,867::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c2603dc4-923d-4c10-9890-95e7d141563a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:43:21,867::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:43:21,868::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:43:21,868::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c2603dc4-923d-4c10-9890-95e7d141563a`::Granted request >Thread-338::DEBUG::2014-05-13 11:43:21,868::task::827::TaskManager.Task::(resourceAcquired) Task=`ac612fd5-ccf9-4b11-8672-51936ad11278`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:43:21,868::task::990::TaskManager.Task::(_decref) Task=`ac612fd5-ccf9-4b11-8672-51936ad11278`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:43:21,869::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:43:21,869::task::1185::TaskManager.Task::(prepare) Task=`ac612fd5-ccf9-4b11-8672-51936ad11278`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:43:21,869::task::595::TaskManager.Task::(_updateState) Task=`ac612fd5-ccf9-4b11-8672-51936ad11278`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:43:21,869::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:43:21,870::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:43:21,870::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:43:21,870::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:43:21,870::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:43:21,870::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:43:21,871::task::990::TaskManager.Task::(_decref) Task=`ac612fd5-ccf9-4b11-8672-51936ad11278`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:43:23,419::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:23,499::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365554 s, 28.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:43:24,837::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:43:25,512::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:25,585::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307453 s, 33.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:43:27,598::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:27,672::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312973 s, 32.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:43:27,857::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:43:29,530::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:43:29,538::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000737722 s, 5.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:43:29,685::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:29,777::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0510478 s, 20.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:43:30,881::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:43:31,790::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:31,868::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352098 s, 29.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:43:31,890::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:43:31,890::task::595::TaskManager.Task::(_updateState) Task=`544b7d2d-058a-406c-9621-2bb46fd27749`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:43:31,891::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:43:31,891::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:43:31,891::task::1185::TaskManager.Task::(prepare) Task=`544b7d2d-058a-406c-9621-2bb46fd27749`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:43:31,891::task::595::TaskManager.Task::(_updateState) Task=`544b7d2d-058a-406c-9621-2bb46fd27749`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:43:31,892::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:43:31,892::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:43:31,892::task::990::TaskManager.Task::(_decref) Task=`544b7d2d-058a-406c-9621-2bb46fd27749`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:43:31,900::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:43:31,900::task::595::TaskManager.Task::(_updateState) Task=`f2495fde-1964-4654-a9d0-646a295f1750`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:43:31,900::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:43:31,901::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ec6ca3ae-65f7-4e75-9cdc-5efba3c83e48`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:43:31,901::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:43:31,901::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:43:31,901::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ec6ca3ae-65f7-4e75-9cdc-5efba3c83e48`::Granted request >Thread-338::DEBUG::2014-05-13 11:43:31,902::task::827::TaskManager.Task::(resourceAcquired) Task=`f2495fde-1964-4654-a9d0-646a295f1750`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:43:31,902::task::990::TaskManager.Task::(_decref) Task=`f2495fde-1964-4654-a9d0-646a295f1750`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:43:31,903::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:43:31,903::task::1185::TaskManager.Task::(prepare) Task=`f2495fde-1964-4654-a9d0-646a295f1750`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:43:31,903::task::595::TaskManager.Task::(_updateState) Task=`f2495fde-1964-4654-a9d0-646a295f1750`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:43:31,903::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:43:31,903::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:43:31,904::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:43:31,904::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:43:31,904::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:43:31,904::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:43:31,904::task::990::TaskManager.Task::(_decref) Task=`f2495fde-1964-4654-a9d0-646a295f1750`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:43:33,881::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:43:33,900::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:43:33,957::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0325976 s, 31.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:43:35,969::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:36,050::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0367448 s, 27.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:43:36,869::task::595::TaskManager.Task::(_updateState) Task=`6a76d9a9-99ec-466c-a67b-b1fa052067ae`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:43:36,870::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:43:36,870::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000737722', 'lastCheck': '7.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:43:36,870::task::1185::TaskManager.Task::(prepare) Task=`6a76d9a9-99ec-466c-a67b-b1fa052067ae`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000737722', 'lastCheck': '7.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:43:36,870::task::595::TaskManager.Task::(_updateState) Task=`6a76d9a9-99ec-466c-a67b-b1fa052067ae`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:43:36,871::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:43:36,871::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:43:36,871::task::990::TaskManager.Task::(_decref) Task=`6a76d9a9-99ec-466c-a67b-b1fa052067ae`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:43:36,919::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:43:38,063::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:38,147::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0411734 s, 24.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:43:39,545::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:43:39,553::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000698117 s, 5.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:43:39,937::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:43:40,160::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:40,241::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.03829 s, 26.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:43:41,922::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [206acf2a] >Thread-13::DEBUG::2014-05-13 11:43:41,923::task::595::TaskManager.Task::(_updateState) Task=`beab5a8d-029d-4162-b1f4-19161d21be44`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:43:41,923::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:43:41,923::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:43:41,924::task::1185::TaskManager.Task::(prepare) Task=`beab5a8d-029d-4162-b1f4-19161d21be44`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:43:41,924::task::595::TaskManager.Task::(_updateState) Task=`beab5a8d-029d-4162-b1f4-19161d21be44`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:43:41,924::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:43:41,924::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:43:41,924::task::990::TaskManager.Task::(_decref) Task=`beab5a8d-029d-4162-b1f4-19161d21be44`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:43:41,929::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [206acf2a] >Thread-338::DEBUG::2014-05-13 11:43:41,930::task::595::TaskManager.Task::(_updateState) Task=`d30ec527-1250-41fb-b546-569d5d3492cd`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:43:41,930::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:43:41,931::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e34ca7a7-220d-4eea-8f53-291129ce2aa8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:43:41,931::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:43:41,931::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:43:41,931::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e34ca7a7-220d-4eea-8f53-291129ce2aa8`::Granted request >Thread-338::DEBUG::2014-05-13 11:43:41,932::task::827::TaskManager.Task::(resourceAcquired) Task=`d30ec527-1250-41fb-b546-569d5d3492cd`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:43:41,932::task::990::TaskManager.Task::(_decref) Task=`d30ec527-1250-41fb-b546-569d5d3492cd`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:43:41,933::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:43:41,933::task::1185::TaskManager.Task::(prepare) Task=`d30ec527-1250-41fb-b546-569d5d3492cd`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:43:41,933::task::595::TaskManager.Task::(_updateState) Task=`d30ec527-1250-41fb-b546-569d5d3492cd`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:43:41,933::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:43:41,933::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:43:41,934::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:43:41,934::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:43:41,934::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:43:41,934::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:43:41,934::task::990::TaskManager.Task::(_decref) Task=`d30ec527-1250-41fb-b546-569d5d3492cd`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:43:42,254::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:42,328::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0308103 s, 33.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:43:42,955::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:43:44,341::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:44,430::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.045794 s, 22.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:43:45,982::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:43:46,443::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:46,522::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359805 s, 28.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:43:48,534::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:48,610::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317777 s, 32.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:43:49,001::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:43:49,560::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:43:49,567::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000578221 s, 7.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:43:50,623::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:50,701::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353251 s, 29.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:43:51,952::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:43:51,953::task::595::TaskManager.Task::(_updateState) Task=`080ae453-3e61-4565-9d79-45cd79c141cf`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:43:51,953::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:43:51,953::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:43:51,954::task::1185::TaskManager.Task::(prepare) Task=`080ae453-3e61-4565-9d79-45cd79c141cf`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:43:51,954::task::595::TaskManager.Task::(_updateState) Task=`080ae453-3e61-4565-9d79-45cd79c141cf`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:43:51,954::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:43:51,954::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:43:51,954::task::990::TaskManager.Task::(_decref) Task=`080ae453-3e61-4565-9d79-45cd79c141cf`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:43:51,958::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:43:51,959::task::595::TaskManager.Task::(_updateState) Task=`90316e2a-0f55-485e-919a-751c4795becf`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:43:51,959::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:43:51,959::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c5258d9e-2238-4886-aa37-b46b787beb07`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:43:51,960::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:43:51,960::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:43:51,960::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c5258d9e-2238-4886-aa37-b46b787beb07`::Granted request >Thread-338::DEBUG::2014-05-13 11:43:51,960::task::827::TaskManager.Task::(resourceAcquired) Task=`90316e2a-0f55-485e-919a-751c4795becf`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:43:51,960::task::990::TaskManager.Task::(_decref) Task=`90316e2a-0f55-485e-919a-751c4795becf`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:43:51,961::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:43:51,961::task::1185::TaskManager.Task::(prepare) Task=`90316e2a-0f55-485e-919a-751c4795becf`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:43:51,962::task::595::TaskManager.Task::(_updateState) Task=`90316e2a-0f55-485e-919a-751c4795becf`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:43:51,962::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:43:51,962::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:43:51,962::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:43:51,962::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:43:51,963::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:43:51,963::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:43:51,963::task::990::TaskManager.Task::(_decref) Task=`90316e2a-0f55-485e-919a-751c4795becf`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:43:52,018::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:43:52,033::task::595::TaskManager.Task::(_updateState) Task=`e8cf85d6-6d36-4401-ab7c-f7c9f3d8d5b2`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:43:52,034::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:43:52,034::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000578221', 'lastCheck': '2.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:43:52,034::task::1185::TaskManager.Task::(prepare) Task=`e8cf85d6-6d36-4401-ab7c-f7c9f3d8d5b2`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000578221', 'lastCheck': '2.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:43:52,034::task::595::TaskManager.Task::(_updateState) Task=`e8cf85d6-6d36-4401-ab7c-f7c9f3d8d5b2`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:43:52,034::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:43:52,034::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:43:52,035::task::990::TaskManager.Task::(_decref) Task=`e8cf85d6-6d36-4401-ab7c-f7c9f3d8d5b2`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:43:52,714::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:52,793::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363251 s, 28.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:43:54,806::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:54,880::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314521 s, 32.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:43:55,040::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:43:56,893::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:56,973::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361225 s, 28.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:43:58,059::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:43:58,986::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:43:59,075::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0462289 s, 22.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:43:59,574::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:43:59,582::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000749161 s, 5.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:44:01,077::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:44:01,088::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:01,169::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0326681 s, 31.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:44:01,981::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2555e502] >Thread-13::DEBUG::2014-05-13 11:44:01,982::task::595::TaskManager.Task::(_updateState) Task=`f13c4b09-60f5-43d5-9e39-a0ff7048ed3f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:44:01,982::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:44:01,983::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:44:01,983::task::1185::TaskManager.Task::(prepare) Task=`f13c4b09-60f5-43d5-9e39-a0ff7048ed3f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:44:01,983::task::595::TaskManager.Task::(_updateState) Task=`f13c4b09-60f5-43d5-9e39-a0ff7048ed3f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:44:01,983::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:44:01,983::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:44:01,984::task::990::TaskManager.Task::(_decref) Task=`f13c4b09-60f5-43d5-9e39-a0ff7048ed3f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:44:01,989::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2555e502] >Thread-338::DEBUG::2014-05-13 11:44:01,989::task::595::TaskManager.Task::(_updateState) Task=`865ee8b4-00b3-4b4f-8f67-bd92e3de0162`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:44:01,989::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:44:01,990::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e2cce049-b74e-4077-a0e2-21120b79c8da`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:44:01,990::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:44:01,990::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:44:01,990::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e2cce049-b74e-4077-a0e2-21120b79c8da`::Granted request >Thread-338::DEBUG::2014-05-13 11:44:01,991::task::827::TaskManager.Task::(resourceAcquired) Task=`865ee8b4-00b3-4b4f-8f67-bd92e3de0162`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:44:01,991::task::990::TaskManager.Task::(_decref) Task=`865ee8b4-00b3-4b4f-8f67-bd92e3de0162`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:44:01,992::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:44:01,992::task::1185::TaskManager.Task::(prepare) Task=`865ee8b4-00b3-4b4f-8f67-bd92e3de0162`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:44:01,992::task::595::TaskManager.Task::(_updateState) Task=`865ee8b4-00b3-4b4f-8f67-bd92e3de0162`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:44:01,992::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:44:01,992::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:44:01,993::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:44:01,993::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:44:01,993::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:44:01,993::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:44:01,993::task::990::TaskManager.Task::(_decref) Task=`865ee8b4-00b3-4b4f-8f67-bd92e3de0162`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:44:03,193::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:03,269::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318933 s, 32.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:44:04,097::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:44:05,282::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:05,362::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359215 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:44:07,115::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:44:07,173::task::595::TaskManager.Task::(_updateState) Task=`1befd05b-3a6f-4c0b-9b48-d716bebdbcfd`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:44:07,173::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:44:07,173::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000749161', 'lastCheck': '7.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:44:07,173::task::1185::TaskManager.Task::(prepare) Task=`1befd05b-3a6f-4c0b-9b48-d716bebdbcfd`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000749161', 'lastCheck': '7.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:44:07,174::task::595::TaskManager.Task::(_updateState) Task=`1befd05b-3a6f-4c0b-9b48-d716bebdbcfd`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:44:07,174::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:44:07,174::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:44:07,174::task::990::TaskManager.Task::(_decref) Task=`1befd05b-3a6f-4c0b-9b48-d716bebdbcfd`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:44:07,375::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:07,455::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0375902 s, 27.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:44:09,468::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:09,545::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0324385 s, 31.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:44:09,589::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:44:09,597::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000642962 s, 6.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:44:10,137::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:44:11,558::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:11,637::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362967 s, 28.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:44:12,013::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:44:12,013::task::595::TaskManager.Task::(_updateState) Task=`52bea3df-4ffe-4ee8-9e4a-b60a719a348b`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:44:12,013::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:44:12,014::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:44:12,014::task::1185::TaskManager.Task::(prepare) Task=`52bea3df-4ffe-4ee8-9e4a-b60a719a348b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:44:12,014::task::595::TaskManager.Task::(_updateState) Task=`52bea3df-4ffe-4ee8-9e4a-b60a719a348b`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:44:12,014::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:44:12,014::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:44:12,014::task::990::TaskManager.Task::(_decref) Task=`52bea3df-4ffe-4ee8-9e4a-b60a719a348b`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:44:12,019::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:44:12,020::task::595::TaskManager.Task::(_updateState) Task=`8aa2959f-4e19-44ba-a61a-3f3cad82262a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:44:12,020::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:44:12,020::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6b2259cc-30ec-4a5a-99e7-1637d8997bed`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:44:12,021::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:44:12,021::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:44:12,021::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6b2259cc-30ec-4a5a-99e7-1637d8997bed`::Granted request >Thread-338::DEBUG::2014-05-13 11:44:12,022::task::827::TaskManager.Task::(resourceAcquired) Task=`8aa2959f-4e19-44ba-a61a-3f3cad82262a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:44:12,023::task::990::TaskManager.Task::(_decref) Task=`8aa2959f-4e19-44ba-a61a-3f3cad82262a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:44:12,023::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:44:12,023::task::1185::TaskManager.Task::(prepare) Task=`8aa2959f-4e19-44ba-a61a-3f3cad82262a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:44:12,024::task::595::TaskManager.Task::(_updateState) Task=`8aa2959f-4e19-44ba-a61a-3f3cad82262a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:44:12,024::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:44:12,024::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:44:12,024::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:44:12,024::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:44:12,025::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:44:12,025::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:44:12,025::task::990::TaskManager.Task::(_decref) Task=`8aa2959f-4e19-44ba-a61a-3f3cad82262a`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:44:13,157::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:44:13,650::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:14,005::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.310907 s, 3.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:44:16,018::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:16,092::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319985 s, 32.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:44:16,176::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:44:18,105::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:18,179::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309566 s, 33.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:44:19,195::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:44:19,604::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:44:19,612::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000587823 s, 7.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:44:20,192::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:20,272::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365422 s, 28.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:44:22,043::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:44:22,044::task::595::TaskManager.Task::(_updateState) Task=`4a742f81-40bf-4ebe-90ec-072ba19ccd8a`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:44:22,044::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:44:22,044::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:44:22,045::task::1185::TaskManager.Task::(prepare) Task=`4a742f81-40bf-4ebe-90ec-072ba19ccd8a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:44:22,045::task::595::TaskManager.Task::(_updateState) Task=`4a742f81-40bf-4ebe-90ec-072ba19ccd8a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:44:22,045::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:44:22,045::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:44:22,045::task::990::TaskManager.Task::(_decref) Task=`4a742f81-40bf-4ebe-90ec-072ba19ccd8a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:44:22,050::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:44:22,050::task::595::TaskManager.Task::(_updateState) Task=`b88132ca-ef49-4975-871e-54e28b9784e9`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:44:22,050::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:44:22,051::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8f59d506-511d-4811-a7d1-9b1b1d4ad44e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:44:22,051::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:44:22,051::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:44:22,051::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8f59d506-511d-4811-a7d1-9b1b1d4ad44e`::Granted request >Thread-338::DEBUG::2014-05-13 11:44:22,052::task::827::TaskManager.Task::(resourceAcquired) Task=`b88132ca-ef49-4975-871e-54e28b9784e9`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:44:22,052::task::990::TaskManager.Task::(_decref) Task=`b88132ca-ef49-4975-871e-54e28b9784e9`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:44:22,053::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:44:22,053::task::1185::TaskManager.Task::(prepare) Task=`b88132ca-ef49-4975-871e-54e28b9784e9`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:44:22,053::task::595::TaskManager.Task::(_updateState) Task=`b88132ca-ef49-4975-871e-54e28b9784e9`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:44:22,053::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:44:22,053::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:44:22,054::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:44:22,054::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:44:22,054::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:44:22,054::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:44:22,054::task::990::TaskManager.Task::(_decref) Task=`b88132ca-ef49-4975-871e-54e28b9784e9`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:44:22,213::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:44:22,285::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 11:44:22,318::task::595::TaskManager.Task::(_updateState) Task=`c0389d2a-7944-49a0-b017-24f924269c66`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:44:22,320::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:44:22,321::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000587823', 'lastCheck': '2.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:44:22,322::task::1185::TaskManager.Task::(prepare) Task=`c0389d2a-7944-49a0-b017-24f924269c66`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000587823', 'lastCheck': '2.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:44:22,322::task::595::TaskManager.Task::(_updateState) Task=`c0389d2a-7944-49a0-b017-24f924269c66`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:44:22,323::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:44:22,324::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:44:22,325::task::990::TaskManager.Task::(_decref) Task=`c0389d2a-7944-49a0-b017-24f924269c66`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:44:22,393::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0626629 s, 16.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:44:24,406::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:24,504::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0492537 s, 20.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:44:25,231::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:44:26,516::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:26,599::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0385463 s, 26.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:44:28,249::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:44:28,612::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:28,686::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031387 s, 32.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:44:29,619::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:44:29,627::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000588245 s, 7.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:44:30,699::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:30,775::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317666 s, 32.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:44:31,278::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:44:32,073::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [72471f8f] >Thread-13::DEBUG::2014-05-13 11:44:32,074::task::595::TaskManager.Task::(_updateState) Task=`3a04d5b8-0624-422c-879c-44e9e16c7720`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:44:32,074::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:44:32,075::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:44:32,075::task::1185::TaskManager.Task::(prepare) Task=`3a04d5b8-0624-422c-879c-44e9e16c7720`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:44:32,075::task::595::TaskManager.Task::(_updateState) Task=`3a04d5b8-0624-422c-879c-44e9e16c7720`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:44:32,075::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:44:32,075::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:44:32,075::task::990::TaskManager.Task::(_decref) Task=`3a04d5b8-0624-422c-879c-44e9e16c7720`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:44:32,080::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [72471f8f] >Thread-338::DEBUG::2014-05-13 11:44:32,080::task::595::TaskManager.Task::(_updateState) Task=`d04498e9-4191-4a69-8628-31f784a271d7`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:44:32,080::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:44:32,081::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`968365c4-aff9-4905-9f28-da6f092bb71f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:44:32,081::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:44:32,081::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:44:32,081::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`968365c4-aff9-4905-9f28-da6f092bb71f`::Granted request >Thread-338::DEBUG::2014-05-13 11:44:32,082::task::827::TaskManager.Task::(resourceAcquired) Task=`d04498e9-4191-4a69-8628-31f784a271d7`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:44:32,082::task::990::TaskManager.Task::(_decref) Task=`d04498e9-4191-4a69-8628-31f784a271d7`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:44:32,083::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:44:32,083::task::1185::TaskManager.Task::(prepare) Task=`d04498e9-4191-4a69-8628-31f784a271d7`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:44:32,083::task::595::TaskManager.Task::(_updateState) Task=`d04498e9-4191-4a69-8628-31f784a271d7`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:44:32,083::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:44:32,083::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:44:32,084::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:44:32,084::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:44:32,084::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:44:32,084::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:44:32,084::task::990::TaskManager.Task::(_decref) Task=`d04498e9-4191-4a69-8628-31f784a271d7`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:44:32,788::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:32,868::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365883 s, 28.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:44:34,299::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:44:34,881::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:34,955::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0308507 s, 33.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:44:36,968::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:37,047::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362504 s, 28.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:44:37,317::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:44:37,476::task::595::TaskManager.Task::(_updateState) Task=`93995edd-544d-4d8e-90d6-b7ba60d29c99`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:44:37,477::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:44:37,477::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000588245', 'lastCheck': '7.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:44:37,477::task::1185::TaskManager.Task::(prepare) Task=`93995edd-544d-4d8e-90d6-b7ba60d29c99`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000588245', 'lastCheck': '7.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:44:37,477::task::595::TaskManager.Task::(_updateState) Task=`93995edd-544d-4d8e-90d6-b7ba60d29c99`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:44:37,478::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:44:37,478::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:44:37,478::task::990::TaskManager.Task::(_decref) Task=`93995edd-544d-4d8e-90d6-b7ba60d29c99`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:44:39,060::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:39,457::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.354996 s, 2.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:44:39,634::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:44:39,642::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000661402 s, 6.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:44:40,336::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:44:41,471::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:41,561::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362602 s, 28.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:44:42,102::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:44:42,102::task::595::TaskManager.Task::(_updateState) Task=`c12f94bc-1545-431b-9e9d-bb1dbc3a3a45`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:44:42,102::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:44:42,103::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:44:42,103::task::1185::TaskManager.Task::(prepare) Task=`c12f94bc-1545-431b-9e9d-bb1dbc3a3a45`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:44:42,103::task::595::TaskManager.Task::(_updateState) Task=`c12f94bc-1545-431b-9e9d-bb1dbc3a3a45`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:44:42,103::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:44:42,103::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:44:42,103::task::990::TaskManager.Task::(_decref) Task=`c12f94bc-1545-431b-9e9d-bb1dbc3a3a45`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:44:42,108::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:44:42,108::task::595::TaskManager.Task::(_updateState) Task=`46f133bf-378e-41af-9ed3-b2edeacb3849`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:44:42,108::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:44:42,108::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`91b06f5c-6022-4a0d-b815-ae7f6a4a9d61`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:44:42,109::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:44:42,109::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:44:42,109::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`91b06f5c-6022-4a0d-b815-ae7f6a4a9d61`::Granted request >Thread-338::DEBUG::2014-05-13 11:44:42,109::task::827::TaskManager.Task::(resourceAcquired) Task=`46f133bf-378e-41af-9ed3-b2edeacb3849`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:44:42,109::task::990::TaskManager.Task::(_decref) Task=`46f133bf-378e-41af-9ed3-b2edeacb3849`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:44:42,110::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:44:42,110::task::1185::TaskManager.Task::(prepare) Task=`46f133bf-378e-41af-9ed3-b2edeacb3849`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:44:42,110::task::595::TaskManager.Task::(_updateState) Task=`46f133bf-378e-41af-9ed3-b2edeacb3849`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:44:42,111::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:44:42,111::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:44:42,111::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:44:42,111::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:44:42,111::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:44:42,111::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:44:42,112::task::990::TaskManager.Task::(_decref) Task=`46f133bf-378e-41af-9ed3-b2edeacb3849`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:44:43,355::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:44:43,575::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:43,649::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310155 s, 33.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:44:45,662::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:45,737::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310816 s, 32.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:44:46,376::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:44:47,750::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:47,830::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0372381 s, 27.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:44:49,395::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:44:49,649::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:44:49,658::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00184318 s, 2.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:44:49,843::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:49,922::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358359 s, 28.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:44:51,935::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:52,028::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0487897 s, 21.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:44:52,128::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2ec76adc] >Thread-13::DEBUG::2014-05-13 11:44:52,129::task::595::TaskManager.Task::(_updateState) Task=`a6ee1976-14dc-4a9c-bd1b-578eda1ebf97`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:44:52,129::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:44:52,129::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:44:52,129::task::1185::TaskManager.Task::(prepare) Task=`a6ee1976-14dc-4a9c-bd1b-578eda1ebf97`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:44:52,130::task::595::TaskManager.Task::(_updateState) Task=`a6ee1976-14dc-4a9c-bd1b-578eda1ebf97`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:44:52,130::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:44:52,130::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:44:52,130::task::990::TaskManager.Task::(_decref) Task=`a6ee1976-14dc-4a9c-bd1b-578eda1ebf97`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:44:52,135::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2ec76adc] >Thread-338::DEBUG::2014-05-13 11:44:52,135::task::595::TaskManager.Task::(_updateState) Task=`40ddc3e2-9da6-4ee1-94e9-883022736e19`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:44:52,136::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:44:52,136::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1f4fea6c-99ca-4b91-893c-22c4af003f40`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:44:52,136::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:44:52,136::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:44:52,137::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1f4fea6c-99ca-4b91-893c-22c4af003f40`::Granted request >Thread-338::DEBUG::2014-05-13 11:44:52,137::task::827::TaskManager.Task::(resourceAcquired) Task=`40ddc3e2-9da6-4ee1-94e9-883022736e19`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:44:52,137::task::990::TaskManager.Task::(_decref) Task=`40ddc3e2-9da6-4ee1-94e9-883022736e19`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:44:52,138::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:44:52,138::task::1185::TaskManager.Task::(prepare) Task=`40ddc3e2-9da6-4ee1-94e9-883022736e19`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:44:52,138::task::595::TaskManager.Task::(_updateState) Task=`40ddc3e2-9da6-4ee1-94e9-883022736e19`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:44:52,139::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:44:52,139::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:44:52,139::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:44:52,139::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:44:52,139::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:44:52,139::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:44:52,140::task::990::TaskManager.Task::(_decref) Task=`40ddc3e2-9da6-4ee1-94e9-883022736e19`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:44:52,413::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:44:52,610::task::595::TaskManager.Task::(_updateState) Task=`a33df085-c06e-432c-86bf-d7d87115319d`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:44:52,610::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:44:52,610::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00184318', 'lastCheck': '2.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:44:52,610::task::1185::TaskManager.Task::(prepare) Task=`a33df085-c06e-432c-86bf-d7d87115319d`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00184318', 'lastCheck': '2.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:44:52,610::task::595::TaskManager.Task::(_updateState) Task=`a33df085-c06e-432c-86bf-d7d87115319d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:44:52,611::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:44:52,611::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:44:52,611::task::990::TaskManager.Task::(_decref) Task=`a33df085-c06e-432c-86bf-d7d87115319d`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:44:54,042::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:54,121::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0369139 s, 27.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:44:55,435::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:44:56,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:56,216::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0386157 s, 26.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:44:58,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:44:58,304::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321871 s, 31.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:44:58,455::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:44:59,662::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 41830dad-f012-4b4d-b636-863f66d5ca4b >Thread-322::DEBUG::2014-05-13 11:44:59,663::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-322::DEBUG::2014-05-13 11:44:59,665::blockSD::336::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:44:59,673::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000636726 s, 3.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:44:59,673::misc::287::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000636726 s, 3.2 MB/s'], size: 2048 >Thread-322::DEBUG::2014-05-13 11:44:59,673::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-322::WARNING::2014-05-13 11:44:59,674::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-322::DEBUG::2014-05-13 11:44:59,674::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-322::DEBUG::2014-05-13 11:44:59,674::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:44:59,675::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:44:59,675::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:44:59,675::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:44:59,675::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:44:59,676::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 11:44:59,941::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:44:59,943::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:44:59,943::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'LOCKPOLICY=', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', 'DESCRIPTION=ISCSI', 'LOGBLKSIZE=512', 'TYPE=ISCSI', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'PHYBLKSIZE=512', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', 'CLASS=Data', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'POOL_DESCRIPTION=DC_2', 'POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'MASTER_VERSION=1', 'ROLE=Master', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', '_SHA_CKSUM=28bb2d7cc60e0d779eb69fe93d050c16c769f069', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0'] >Thread-322::INFO::2014-05-13 11:44:59,944::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_imageNS already registered >Thread-322::INFO::2014-05-13 11:44:59,945::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_volumeNS already registered >Thread-322::INFO::2014-05-13 11:44:59,945::blockSD::452::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_lvmActivationNS already registered >Thread-322::DEBUG::2014-05-13 11:44:59,946::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 11:45:00,153::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:45:00,156::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:45:00,164::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000546399 s, 7.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:45:00,317::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:00,397::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0368066 s, 27.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:45:01,475::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:45:02,158::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:45:02,159::task::595::TaskManager.Task::(_updateState) Task=`537155a2-bb5a-4a71-b72d-8396928fd13a`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:45:02,159::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:45:02,159::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:45:02,159::task::1185::TaskManager.Task::(prepare) Task=`537155a2-bb5a-4a71-b72d-8396928fd13a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:45:02,160::task::595::TaskManager.Task::(_updateState) Task=`537155a2-bb5a-4a71-b72d-8396928fd13a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:45:02,160::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:45:02,160::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:45:02,160::task::990::TaskManager.Task::(_decref) Task=`537155a2-bb5a-4a71-b72d-8396928fd13a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:45:02,165::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:45:02,165::task::595::TaskManager.Task::(_updateState) Task=`ba7c45c7-e1ed-42f3-a31c-4dac99ad8b24`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:45:02,166::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:45:02,166::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e04ebca4-2d7b-4493-9874-4844ce5fffe0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:45:02,166::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:45:02,166::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:45:02,167::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e04ebca4-2d7b-4493-9874-4844ce5fffe0`::Granted request >Thread-338::DEBUG::2014-05-13 11:45:02,167::task::827::TaskManager.Task::(resourceAcquired) Task=`ba7c45c7-e1ed-42f3-a31c-4dac99ad8b24`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:45:02,167::task::990::TaskManager.Task::(_decref) Task=`ba7c45c7-e1ed-42f3-a31c-4dac99ad8b24`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:45:02,168::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:45:02,168::task::1185::TaskManager.Task::(prepare) Task=`ba7c45c7-e1ed-42f3-a31c-4dac99ad8b24`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:45:02,168::task::595::TaskManager.Task::(_updateState) Task=`ba7c45c7-e1ed-42f3-a31c-4dac99ad8b24`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:45:02,168::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:45:02,169::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:45:02,169::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:45:02,169::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:45:02,169::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:45:02,169::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:45:02,170::task::990::TaskManager.Task::(_decref) Task=`ba7c45c7-e1ed-42f3-a31c-4dac99ad8b24`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:45:02,410::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:02,489::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355339 s, 28.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:45:04,494::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:45:04,502::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:04,577::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0326142 s, 31.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:45:06,590::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:06,691::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0462691 s, 22.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:45:07,513::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:45:07,745::task::595::TaskManager.Task::(_updateState) Task=`3ba444bf-ad76-4049-b471-ee322bb3bd92`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:45:07,745::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:45:07,745::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000546399', 'lastCheck': '7.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:45:07,745::task::1185::TaskManager.Task::(prepare) Task=`3ba444bf-ad76-4049-b471-ee322bb3bd92`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000546399', 'lastCheck': '7.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:45:07,745::task::595::TaskManager.Task::(_updateState) Task=`3ba444bf-ad76-4049-b471-ee322bb3bd92`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:45:07,746::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:45:07,746::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:45:07,746::task::990::TaskManager.Task::(_decref) Task=`3ba444bf-ad76-4049-b471-ee322bb3bd92`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:45:08,704::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:08,781::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035717 s, 28.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:45:10,172::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:45:10,179::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000633041 s, 6.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:45:10,532::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:45:10,794::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:10,879::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0416579 s, 24.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:45:12,190::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:45:12,190::task::595::TaskManager.Task::(_updateState) Task=`c42130d9-a15d-446a-829a-691095dc93dc`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:45:12,190::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:45:12,191::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:45:12,191::task::1185::TaskManager.Task::(prepare) Task=`c42130d9-a15d-446a-829a-691095dc93dc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:45:12,191::task::595::TaskManager.Task::(_updateState) Task=`c42130d9-a15d-446a-829a-691095dc93dc`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:45:12,191::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:45:12,191::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:45:12,191::task::990::TaskManager.Task::(_decref) Task=`c42130d9-a15d-446a-829a-691095dc93dc`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:45:12,197::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:45:12,198::task::595::TaskManager.Task::(_updateState) Task=`784646c7-1e6e-451d-a55d-0ba775dcfe84`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:45:12,198::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:45:12,198::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a4098502-49f2-47f2-931d-d4093e403c9c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:45:12,198::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:45:12,199::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:45:12,199::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a4098502-49f2-47f2-931d-d4093e403c9c`::Granted request >Thread-338::DEBUG::2014-05-13 11:45:12,199::task::827::TaskManager.Task::(resourceAcquired) Task=`784646c7-1e6e-451d-a55d-0ba775dcfe84`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:45:12,199::task::990::TaskManager.Task::(_decref) Task=`784646c7-1e6e-451d-a55d-0ba775dcfe84`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:45:12,200::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:45:12,200::task::1185::TaskManager.Task::(prepare) Task=`784646c7-1e6e-451d-a55d-0ba775dcfe84`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:45:12,200::task::595::TaskManager.Task::(_updateState) Task=`784646c7-1e6e-451d-a55d-0ba775dcfe84`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:45:12,200::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:45:12,200::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:45:12,201::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:45:12,201::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:45:12,201::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:45:12,201::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:45:12,201::task::990::TaskManager.Task::(_decref) Task=`784646c7-1e6e-451d-a55d-0ba775dcfe84`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:45:12,892::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:12,971::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0377018 s, 27.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:45:13,560::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:45:14,984::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:15,065::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364171 s, 28.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:45:16,582::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:45:17,078::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:17,158::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0375062 s, 27.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:45:19,171::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:19,248::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0350204 s, 29.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:45:19,601::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:45:20,186::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:45:20,194::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000767405 s, 5.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:45:21,261::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:21,346::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0417041 s, 24.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:45:22,222::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3c71a9dd] >Thread-13::DEBUG::2014-05-13 11:45:22,237::task::595::TaskManager.Task::(_updateState) Task=`38e1fb51-c5bb-4dc5-bb11-ff6d1b3be750`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:45:22,238::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:45:22,238::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:45:22,238::task::1185::TaskManager.Task::(prepare) Task=`38e1fb51-c5bb-4dc5-bb11-ff6d1b3be750`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:45:22,238::task::595::TaskManager.Task::(_updateState) Task=`38e1fb51-c5bb-4dc5-bb11-ff6d1b3be750`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:45:22,238::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:45:22,239::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:45:22,239::task::990::TaskManager.Task::(_decref) Task=`38e1fb51-c5bb-4dc5-bb11-ff6d1b3be750`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:45:22,245::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3c71a9dd] >Thread-338::DEBUG::2014-05-13 11:45:22,245::task::595::TaskManager.Task::(_updateState) Task=`99c79279-1f9f-4145-9e8d-73e35abbda9f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:45:22,245::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:45:22,246::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0d67892b-3e71-4551-a429-7a23c0615640`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:45:22,247::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:45:22,247::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:45:22,247::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0d67892b-3e71-4551-a429-7a23c0615640`::Granted request >Thread-338::DEBUG::2014-05-13 11:45:22,247::task::827::TaskManager.Task::(resourceAcquired) Task=`99c79279-1f9f-4145-9e8d-73e35abbda9f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:45:22,248::task::990::TaskManager.Task::(_decref) Task=`99c79279-1f9f-4145-9e8d-73e35abbda9f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:45:22,248::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:45:22,248::task::1185::TaskManager.Task::(prepare) Task=`99c79279-1f9f-4145-9e8d-73e35abbda9f`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:45:22,249::task::595::TaskManager.Task::(_updateState) Task=`99c79279-1f9f-4145-9e8d-73e35abbda9f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:45:22,249::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:45:22,249::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:45:22,249::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:45:22,249::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:45:22,250::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:45:22,250::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:45:22,250::task::990::TaskManager.Task::(_decref) Task=`99c79279-1f9f-4145-9e8d-73e35abbda9f`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:45:22,623::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:45:22,898::task::595::TaskManager.Task::(_updateState) Task=`e2d4bce3-0d35-40b1-8d95-21fb8504b69e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:45:22,898::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:45:22,898::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000767405', 'lastCheck': '2.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:45:22,898::task::1185::TaskManager.Task::(prepare) Task=`e2d4bce3-0d35-40b1-8d95-21fb8504b69e`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000767405', 'lastCheck': '2.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:45:22,898::task::595::TaskManager.Task::(_updateState) Task=`e2d4bce3-0d35-40b1-8d95-21fb8504b69e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:45:22,899::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:45:22,899::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:45:22,899::task::990::TaskManager.Task::(_decref) Task=`e2d4bce3-0d35-40b1-8d95-21fb8504b69e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:45:23,359::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:23,450::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0495143 s, 20.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:45:25,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:25,535::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0302835 s, 33.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:45:25,642::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:45:27,548::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:27,622::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322666 s, 31.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:45:28,661::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:45:29,635::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:29,714::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036072 s, 28.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:45:30,201::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:45:30,209::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000638693 s, 6.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:45:31,690::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:45:31,727::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:31,814::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0421596 s, 24.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:45:32,270::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:45:32,270::task::595::TaskManager.Task::(_updateState) Task=`31c13921-2bf3-42a5-890f-25a01c65a4c3`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:45:32,270::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:45:32,271::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:45:32,271::task::1185::TaskManager.Task::(prepare) Task=`31c13921-2bf3-42a5-890f-25a01c65a4c3`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:45:32,271::task::595::TaskManager.Task::(_updateState) Task=`31c13921-2bf3-42a5-890f-25a01c65a4c3`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:45:32,271::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:45:32,271::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:45:32,271::task::990::TaskManager.Task::(_decref) Task=`31c13921-2bf3-42a5-890f-25a01c65a4c3`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:45:32,276::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:45:32,276::task::595::TaskManager.Task::(_updateState) Task=`b04fa6c2-345f-4315-9f80-537e1ad04a87`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:45:32,276::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:45:32,277::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2c084143-6a21-4e2c-be48-959bda2d6545`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:45:32,277::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:45:32,277::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:45:32,277::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2c084143-6a21-4e2c-be48-959bda2d6545`::Granted request >Thread-338::DEBUG::2014-05-13 11:45:32,278::task::827::TaskManager.Task::(resourceAcquired) Task=`b04fa6c2-345f-4315-9f80-537e1ad04a87`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:45:32,278::task::990::TaskManager.Task::(_decref) Task=`b04fa6c2-345f-4315-9f80-537e1ad04a87`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:45:32,278::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:45:32,279::task::1185::TaskManager.Task::(prepare) Task=`b04fa6c2-345f-4315-9f80-537e1ad04a87`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:45:32,279::task::595::TaskManager.Task::(_updateState) Task=`b04fa6c2-345f-4315-9f80-537e1ad04a87`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:45:32,279::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:45:32,279::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:45:32,279::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:45:32,280::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:45:32,280::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:45:32,280::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:45:32,280::task::990::TaskManager.Task::(_decref) Task=`b04fa6c2-345f-4315-9f80-537e1ad04a87`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:45:33,826::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:33,898::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307081 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:45:34,709::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:45:35,911::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:35,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0459739 s, 22.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:45:37,732::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:45:38,013::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 11:45:38,037::task::595::TaskManager.Task::(_updateState) Task=`c282a7f6-f78a-4ab2-b05e-35d5f6d9d743`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:45:38,039::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:45:38,040::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000638693', 'lastCheck': '7.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:45:38,041::task::1185::TaskManager.Task::(prepare) Task=`c282a7f6-f78a-4ab2-b05e-35d5f6d9d743`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000638693', 'lastCheck': '7.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:45:38,043::task::595::TaskManager.Task::(_updateState) Task=`c282a7f6-f78a-4ab2-b05e-35d5f6d9d743`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:45:38,044::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:45:38,045::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:45:38,046::task::990::TaskManager.Task::(_decref) Task=`c282a7f6-f78a-4ab2-b05e-35d5f6d9d743`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:45:38,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0706644 s, 14.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:45:40,139::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:40,211::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031118 s, 32.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:45:40,232::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:45:40,248::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000663193 s, 6.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:45:40,751::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:45:42,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 11:45:42,298::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:45:42,299::task::595::TaskManager.Task::(_updateState) Task=`a61eec84-f3e9-40d0-998b-78dc5682f747`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:45:42,300::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:45:42,300::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:45:42,300::task::1185::TaskManager.Task::(prepare) Task=`a61eec84-f3e9-40d0-998b-78dc5682f747`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:45:42,301::task::595::TaskManager.Task::(_updateState) Task=`a61eec84-f3e9-40d0-998b-78dc5682f747`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:45:42,301::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:45:42,301::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:45:42,301::task::990::TaskManager.Task::(_decref) Task=`a61eec84-f3e9-40d0-998b-78dc5682f747`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:45:42,307::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:45:42,308::task::595::TaskManager.Task::(_updateState) Task=`523fd181-68f6-4877-9266-7d18a645ccb8`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:45:42,308::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:45:42,308::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`66a7f3dc-9cfa-4873-a5a5-36cbccaaa65f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:45:42,309::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:45:42,309::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:45:42,309::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`66a7f3dc-9cfa-4873-a5a5-36cbccaaa65f`::Granted request >Thread-338::DEBUG::2014-05-13 11:45:42,310::task::827::TaskManager.Task::(resourceAcquired) Task=`523fd181-68f6-4877-9266-7d18a645ccb8`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:45:42,310::task::990::TaskManager.Task::(_decref) Task=`523fd181-68f6-4877-9266-7d18a645ccb8`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:45:42,311::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:45:42,311::task::1185::TaskManager.Task::(prepare) Task=`523fd181-68f6-4877-9266-7d18a645ccb8`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:45:42,312::task::595::TaskManager.Task::(_updateState) Task=`523fd181-68f6-4877-9266-7d18a645ccb8`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:45:42,312::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:45:42,312::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:45:42,312::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:45:42,313::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:45:42,313::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:45:42,313::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:45:42,313::task::990::TaskManager.Task::(_decref) Task=`523fd181-68f6-4877-9266-7d18a645ccb8`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:45:42,366::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0435497 s, 23.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:45:43,772::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:45:44,379::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:44,456::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0349648 s, 29.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:45:46,469::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:46,542::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.030625 s, 33.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:45:46,792::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:45:48,555::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:48,638::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0410672 s, 24.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:45:49,811::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:45:50,272::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:45:50,279::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000736944 s, 5.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:45:50,651::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:50,729::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356042 s, 28.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:45:52,333::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6019f20] >Thread-13::DEBUG::2014-05-13 11:45:52,345::task::595::TaskManager.Task::(_updateState) Task=`43a6e425-f8df-438e-8396-4da37f427b51`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:45:52,345::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:45:52,346::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:45:52,346::task::1185::TaskManager.Task::(prepare) Task=`43a6e425-f8df-438e-8396-4da37f427b51`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:45:52,346::task::595::TaskManager.Task::(_updateState) Task=`43a6e425-f8df-438e-8396-4da37f427b51`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:45:52,346::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:45:52,346::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:45:52,347::task::990::TaskManager.Task::(_decref) Task=`43a6e425-f8df-438e-8396-4da37f427b51`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:45:52,352::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6019f20] >Thread-338::DEBUG::2014-05-13 11:45:52,352::task::595::TaskManager.Task::(_updateState) Task=`33966358-1725-48c2-b16b-e35cb6978184`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:45:52,352::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:45:52,353::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`293e1aa3-29a6-4524-a3b5-8d835fc8b9fe`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:45:52,353::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:45:52,353::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:45:52,353::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`293e1aa3-29a6-4524-a3b5-8d835fc8b9fe`::Granted request >Thread-338::DEBUG::2014-05-13 11:45:52,354::task::827::TaskManager.Task::(resourceAcquired) Task=`33966358-1725-48c2-b16b-e35cb6978184`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:45:52,354::task::990::TaskManager.Task::(_decref) Task=`33966358-1725-48c2-b16b-e35cb6978184`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:45:52,354::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:45:52,355::task::1185::TaskManager.Task::(prepare) Task=`33966358-1725-48c2-b16b-e35cb6978184`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:45:52,355::task::595::TaskManager.Task::(_updateState) Task=`33966358-1725-48c2-b16b-e35cb6978184`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:45:52,355::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:45:52,355::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:45:52,355::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:45:52,356::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:45:52,356::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:45:52,356::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:45:52,356::task::990::TaskManager.Task::(_decref) Task=`33966358-1725-48c2-b16b-e35cb6978184`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:45:52,742::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:52,821::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359412 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:45:52,830::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:45:53,193::task::595::TaskManager.Task::(_updateState) Task=`4f0ef24b-ed32-4e96-8dfa-2854aaa4c11a`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:45:53,193::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:45:53,194::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000736944', 'lastCheck': '2.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:45:53,194::task::1185::TaskManager.Task::(prepare) Task=`4f0ef24b-ed32-4e96-8dfa-2854aaa4c11a`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000736944', 'lastCheck': '2.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:45:53,194::task::595::TaskManager.Task::(_updateState) Task=`4f0ef24b-ed32-4e96-8dfa-2854aaa4c11a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:45:53,194::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:45:53,194::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:45:53,195::task::990::TaskManager.Task::(_decref) Task=`4f0ef24b-ed32-4e96-8dfa-2854aaa4c11a`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:45:54,836::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:54,908::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307298 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:45:55,849::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:45:56,921::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:57,000::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357356 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:45:58,867::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:45:59,013::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:45:59,087::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316676 s, 32.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:46:00,286::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:46:00,294::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000604631 s, 6.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:46:01,100::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:46:01,177::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0354308 s, 28.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:46:01,894::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:46:02,376::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:46:02,377::task::595::TaskManager.Task::(_updateState) Task=`b2c9e04b-198f-419a-8e2a-f2c95b19078d`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:46:02,377::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:46:02,377::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:46:02,377::task::1185::TaskManager.Task::(prepare) Task=`b2c9e04b-198f-419a-8e2a-f2c95b19078d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:46:02,378::task::595::TaskManager.Task::(_updateState) Task=`b2c9e04b-198f-419a-8e2a-f2c95b19078d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:46:02,378::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:46:02,378::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:46:02,378::task::990::TaskManager.Task::(_decref) Task=`b2c9e04b-198f-419a-8e2a-f2c95b19078d`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:46:02,382::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:46:02,383::task::595::TaskManager.Task::(_updateState) Task=`aff87a82-2119-43b8-a80b-b29d1f99e771`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:46:02,383::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:46:02,383::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a218452d-9c41-4632-b926-50aee0074a97`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:46:02,383::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:46:02,384::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:46:02,384::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a218452d-9c41-4632-b926-50aee0074a97`::Granted request >Thread-338::DEBUG::2014-05-13 11:46:02,384::task::827::TaskManager.Task::(resourceAcquired) Task=`aff87a82-2119-43b8-a80b-b29d1f99e771`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:46:02,384::task::990::TaskManager.Task::(_decref) Task=`aff87a82-2119-43b8-a80b-b29d1f99e771`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:46:02,385::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:46:02,385::task::1185::TaskManager.Task::(prepare) Task=`aff87a82-2119-43b8-a80b-b29d1f99e771`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:46:02,386::task::595::TaskManager.Task::(_updateState) Task=`aff87a82-2119-43b8-a80b-b29d1f99e771`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:46:02,386::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:46:02,386::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:46:02,386::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:46:02,386::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:46:02,387::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:46:02,387::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:46:02,387::task::990::TaskManager.Task::(_decref) Task=`aff87a82-2119-43b8-a80b-b29d1f99e771`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:46:03,190::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:46:03,275::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0426464 s, 24.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:46:04,913::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:46:05,288::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:46:05,370::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0405283 s, 25.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:46:07,383::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:46:07,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381801 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:46:07,935::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:46:08,334::task::595::TaskManager.Task::(_updateState) Task=`4180f809-7f6a-4310-90e7-7fcb891811b9`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:46:08,334::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:46:08,335::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000604631', 'lastCheck': '8.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:46:08,335::task::1185::TaskManager.Task::(prepare) Task=`4180f809-7f6a-4310-90e7-7fcb891811b9`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000604631', 'lastCheck': '8.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:46:08,335::task::595::TaskManager.Task::(_updateState) Task=`4180f809-7f6a-4310-90e7-7fcb891811b9`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:46:08,335::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:46:08,335::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:46:08,335::task::990::TaskManager.Task::(_decref) Task=`4180f809-7f6a-4310-90e7-7fcb891811b9`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:46:09,476::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:46:09,556::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0371346 s, 27.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:46:10,301::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:46:10,309::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000601448 s, 6.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:46:10,959::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:46:11,569::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:46:11,646::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356787 s, 28.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:46:12,406::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:46:12,407::task::595::TaskManager.Task::(_updateState) Task=`19e6b7ed-3e91-43b1-bd8d-b3526227df9a`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:46:12,407::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:46:12,408::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:46:12,408::task::1185::TaskManager.Task::(prepare) Task=`19e6b7ed-3e91-43b1-bd8d-b3526227df9a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:46:12,408::task::595::TaskManager.Task::(_updateState) Task=`19e6b7ed-3e91-43b1-bd8d-b3526227df9a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:46:12,408::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:46:12,408::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:46:12,408::task::990::TaskManager.Task::(_decref) Task=`19e6b7ed-3e91-43b1-bd8d-b3526227df9a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:46:12,413::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:46:12,413::task::595::TaskManager.Task::(_updateState) Task=`ad63512f-ba8e-46a9-a120-fff5e54395a7`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:46:12,413::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:46:12,413::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`42d47aa7-9bdc-4070-9164-b3f4322e01fe`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:46:12,414::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:46:12,414::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:46:12,414::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`42d47aa7-9bdc-4070-9164-b3f4322e01fe`::Granted request >Thread-338::DEBUG::2014-05-13 11:46:12,414::task::827::TaskManager.Task::(resourceAcquired) Task=`ad63512f-ba8e-46a9-a120-fff5e54395a7`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:46:12,414::task::990::TaskManager.Task::(_decref) Task=`ad63512f-ba8e-46a9-a120-fff5e54395a7`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:46:12,415::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:46:12,415::task::1185::TaskManager.Task::(prepare) Task=`ad63512f-ba8e-46a9-a120-fff5e54395a7`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:46:12,415::task::595::TaskManager.Task::(_updateState) Task=`ad63512f-ba8e-46a9-a120-fff5e54395a7`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:46:12,415::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:46:12,416::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:46:12,416::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:46:12,416::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:46:12,416::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:46:12,416::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:46:12,416::task::990::TaskManager.Task::(_decref) Task=`ad63512f-ba8e-46a9-a120-fff5e54395a7`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:46:13,659::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:46:13,734::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320185 s, 32.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:46:13,977::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:46:15,747::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:46:16,150::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.360353 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:46:17,000::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:46:18,163::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:46:18,237::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309852 s, 33.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:46:20,024::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:46:20,251::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 11:46:20,336::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:46:20,353::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00159032 s, 2.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:46:20,384::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0350277 s, 29.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:46:22,397::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 11:46:22,435::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:46:22,437::task::595::TaskManager.Task::(_updateState) Task=`f397dcef-ef42-4cfe-af77-c0401afe86eb`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:46:22,438::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:46:22,439::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:46:22,440::task::1185::TaskManager.Task::(prepare) Task=`f397dcef-ef42-4cfe-af77-c0401afe86eb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:46:22,441::task::595::TaskManager.Task::(_updateState) Task=`f397dcef-ef42-4cfe-af77-c0401afe86eb`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:46:22,442::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:46:22,443::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:46:22,444::task::990::TaskManager.Task::(_decref) Task=`f397dcef-ef42-4cfe-af77-c0401afe86eb`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:46:22,451::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:46:22,453::task::595::TaskManager.Task::(_updateState) Task=`b077a0fc-6b17-4e4f-b6cf-229ddaa003b7`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:46:22,454::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:46:22,456::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`489bc55c-ac4f-4d4a-82e1-01290e777b38`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:46:22,457::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:46:22,458::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:46:22,459::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`489bc55c-ac4f-4d4a-82e1-01290e777b38`::Granted request >Thread-338::DEBUG::2014-05-13 11:46:22,461::task::827::TaskManager.Task::(resourceAcquired) Task=`b077a0fc-6b17-4e4f-b6cf-229ddaa003b7`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:46:22,462::task::990::TaskManager.Task::(_decref) Task=`b077a0fc-6b17-4e4f-b6cf-229ddaa003b7`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:46:22,466::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:46:22,467::task::1185::TaskManager.Task::(prepare) Task=`b077a0fc-6b17-4e4f-b6cf-229ddaa003b7`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:46:22,468::task::595::TaskManager.Task::(_updateState) Task=`b077a0fc-6b17-4e4f-b6cf-229ddaa003b7`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:46:22,469::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:46:22,470::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:46:22,471::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:46:22,472::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:46:22,473::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:46:22,474::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:46:22,475::task::990::TaskManager.Task::(_decref) Task=`b077a0fc-6b17-4e4f-b6cf-229ddaa003b7`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:46:22,516::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0767222 s, 13.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:46:23,042::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:46:23,479::task::595::TaskManager.Task::(_updateState) Task=`b41074e7-6748-4051-8e35-e9a5ab007bed`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:46:23,479::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:46:23,480::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00159032', 'lastCheck': '3.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:46:23,480::task::1185::TaskManager.Task::(prepare) Task=`b41074e7-6748-4051-8e35-e9a5ab007bed`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00159032', 'lastCheck': '3.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:46:23,480::task::595::TaskManager.Task::(_updateState) Task=`b41074e7-6748-4051-8e35-e9a5ab007bed`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:46:23,480::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:46:23,480::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:46:23,480::task::990::TaskManager.Task::(_decref) Task=`b41074e7-6748-4051-8e35-e9a5ab007bed`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:46:24,529::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:46:24,604::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0330118 s, 31.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:46:26,062::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:46:26,617::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:46:26,697::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0380068 s, 26.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:46:28,709::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:46:28,796::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0444333 s, 23.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:46:29,080::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:46:30,374::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:46:30,382::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000692734 s, 5.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:46:30,809::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:46:30,888::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381701 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:46:32,099::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:46:32,497::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:46:32,498::task::595::TaskManager.Task::(_updateState) Task=`d3971116-3571-4b12-93fc-ab374c45f076`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:46:32,498::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:46:32,499::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:46:32,499::task::1185::TaskManager.Task::(prepare) Task=`d3971116-3571-4b12-93fc-ab374c45f076`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:46:32,499::task::595::TaskManager.Task::(_updateState) Task=`d3971116-3571-4b12-93fc-ab374c45f076`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:46:32,499::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:46:32,499::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:46:32,499::task::990::TaskManager.Task::(_decref) Task=`d3971116-3571-4b12-93fc-ab374c45f076`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:46:32,505::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:46:32,505::task::595::TaskManager.Task::(_updateState) Task=`7b4a6c5a-001c-4cd2-859c-ac0a36142dce`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:46:32,505::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:46:32,506::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`787c156e-1f19-4357-a53a-f56ec323d308`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:46:32,506::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:46:32,506::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:46:32,507::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`787c156e-1f19-4357-a53a-f56ec323d308`::Granted request >Thread-338::DEBUG::2014-05-13 11:46:32,508::task::827::TaskManager.Task::(resourceAcquired) Task=`7b4a6c5a-001c-4cd2-859c-ac0a36142dce`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:46:32,508::task::990::TaskManager.Task::(_decref) Task=`7b4a6c5a-001c-4cd2-859c-ac0a36142dce`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:46:32,509::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:46:32,509::task::1185::TaskManager.Task::(prepare) Task=`7b4a6c5a-001c-4cd2-859c-ac0a36142dce`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:46:32,509::task::595::TaskManager.Task::(_updateState) Task=`7b4a6c5a-001c-4cd2-859c-ac0a36142dce`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:46:32,509::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:46:32,510::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:46:32,510::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:46:32,510::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:46:32,510::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:46:32,510::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:46:32,510::task::990::TaskManager.Task::(_decref) Task=`7b4a6c5a-001c-4cd2-859c-ac0a36142dce`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:46:32,901::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:46:32,981::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.038696 s, 26.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:46:34,994::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:46:35,069::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0332894 s, 30.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:46:35,118::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:46:37,200::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:46:37,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0354689 s, 28.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:46:38,141::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:46:38,613::task::595::TaskManager.Task::(_updateState) Task=`9efd7f20-d15d-44e0-9546-818f473a11ac`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:46:38,614::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:46:38,614::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000692734', 'lastCheck': '8.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:46:38,614::task::1185::TaskManager.Task::(prepare) Task=`9efd7f20-d15d-44e0-9546-818f473a11ac`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000692734', 'lastCheck': '8.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:46:38,614::task::595::TaskManager.Task::(_updateState) Task=`9efd7f20-d15d-44e0-9546-818f473a11ac`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:46:38,614::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:46:38,614::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:46:38,615::task::990::TaskManager.Task::(_decref) Task=`9efd7f20-d15d-44e0-9546-818f473a11ac`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:46:39,291::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:46:39,365::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319074 s, 32.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:46:40,389::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:46:40,397::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000785411 s, 5.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:46:41,166::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:46:41,378::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:46:41,459::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383571 s, 26.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:46:42,530::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:46:42,530::task::595::TaskManager.Task::(_updateState) Task=`da8d9cbe-a6e7-418d-a514-b0b138a12293`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:46:42,530::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:46:42,531::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:46:42,531::task::1185::TaskManager.Task::(prepare) Task=`da8d9cbe-a6e7-418d-a514-b0b138a12293`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:46:42,531::task::595::TaskManager.Task::(_updateState) Task=`da8d9cbe-a6e7-418d-a514-b0b138a12293`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:46:42,531::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:46:42,531::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:46:42,531::task::990::TaskManager.Task::(_decref) Task=`da8d9cbe-a6e7-418d-a514-b0b138a12293`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:46:42,536::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:46:42,536::task::595::TaskManager.Task::(_updateState) Task=`cd8b2912-a376-42bf-96ed-4d94ec3556f2`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:46:42,536::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:46:42,537::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`271cd5cf-434c-4659-8629-01620e016531`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:46:42,537::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:46:42,537::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:46:42,538::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`271cd5cf-434c-4659-8629-01620e016531`::Granted request >Thread-338::DEBUG::2014-05-13 11:46:42,538::task::827::TaskManager.Task::(resourceAcquired) Task=`cd8b2912-a376-42bf-96ed-4d94ec3556f2`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:46:42,538::task::990::TaskManager.Task::(_decref) Task=`cd8b2912-a376-42bf-96ed-4d94ec3556f2`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:46:42,539::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:46:42,539::task::1185::TaskManager.Task::(prepare) Task=`cd8b2912-a376-42bf-96ed-4d94ec3556f2`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:46:42,539::task::595::TaskManager.Task::(_updateState) Task=`cd8b2912-a376-42bf-96ed-4d94ec3556f2`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:46:42,539::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:46:42,539::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:46:42,540::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:46:42,540::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:46:42,540::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:46:42,540::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:46:42,540::task::990::TaskManager.Task::(_decref) Task=`cd8b2912-a376-42bf-96ed-4d94ec3556f2`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:46:43,472::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:46:43,561::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0449077 s, 22.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:46:44,185::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:46:45,574::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:46:45,656::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0394625 s, 25.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:46:47,203::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:46:47,669::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:46:47,754::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379928 s, 27.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:46:49,766::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:46:49,842::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322532 s, 31.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:46:50,222::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:46:50,405::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:46:50,412::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000694367 s, 5.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:46:51,855::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:46:51,928::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.030437 s, 33.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:46:52,559::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:46:52,559::task::595::TaskManager.Task::(_updateState) Task=`757e0aad-5bdf-4c88-b907-126bcab109f2`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:46:52,560::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:46:52,560::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:46:52,560::task::1185::TaskManager.Task::(prepare) Task=`757e0aad-5bdf-4c88-b907-126bcab109f2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:46:52,560::task::595::TaskManager.Task::(_updateState) Task=`757e0aad-5bdf-4c88-b907-126bcab109f2`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:46:52,561::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:46:52,561::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:46:52,561::task::990::TaskManager.Task::(_decref) Task=`757e0aad-5bdf-4c88-b907-126bcab109f2`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:46:52,565::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:46:52,566::task::595::TaskManager.Task::(_updateState) Task=`04a6dbc1-0239-44cf-9991-128dfa1ff96e`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:46:52,566::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:46:52,566::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1b5b0f60-3256-40e1-8170-0d2fd53dbc2e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:46:52,566::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:46:52,567::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:46:52,567::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1b5b0f60-3256-40e1-8170-0d2fd53dbc2e`::Granted request >Thread-338::DEBUG::2014-05-13 11:46:52,567::task::827::TaskManager.Task::(resourceAcquired) Task=`04a6dbc1-0239-44cf-9991-128dfa1ff96e`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:46:52,567::task::990::TaskManager.Task::(_decref) Task=`04a6dbc1-0239-44cf-9991-128dfa1ff96e`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:46:52,568::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:46:52,568::task::1185::TaskManager.Task::(prepare) Task=`04a6dbc1-0239-44cf-9991-128dfa1ff96e`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:46:52,568::task::595::TaskManager.Task::(_updateState) Task=`04a6dbc1-0239-44cf-9991-128dfa1ff96e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:46:52,569::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:46:52,569::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:46:52,569::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:46:52,569::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:46:52,569::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:46:52,570::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:46:52,570::task::990::TaskManager.Task::(_decref) Task=`04a6dbc1-0239-44cf-9991-128dfa1ff96e`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:46:53,245::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:46:53,765::task::595::TaskManager.Task::(_updateState) Task=`d2607527-7209-446d-b5a6-6ad0784dd1bb`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:46:53,766::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:46:53,766::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000694367', 'lastCheck': '3.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:46:53,766::task::1185::TaskManager.Task::(prepare) Task=`d2607527-7209-446d-b5a6-6ad0784dd1bb`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000694367', 'lastCheck': '3.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:46:53,766::task::595::TaskManager.Task::(_updateState) Task=`d2607527-7209-446d-b5a6-6ad0784dd1bb`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:46:53,766::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:46:53,766::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:46:53,767::task::990::TaskManager.Task::(_decref) Task=`d2607527-7209-446d-b5a6-6ad0784dd1bb`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:46:53,941::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:46:54,020::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363326 s, 28.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:46:56,033::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:46:56,110::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0327417 s, 31.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:46:56,265::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:46:58,123::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:46:58,491::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.325387 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:46:59,284::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:47:00,419::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:47:00,427::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000735169 s, 5.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:47:00,504::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:47:00,584::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0378489 s, 27.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:47:02,303::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:47:02,590::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6602059] >Thread-13::DEBUG::2014-05-13 11:47:02,591::task::595::TaskManager.Task::(_updateState) Task=`7d878a6b-7997-4cca-97d2-83f31f4d0e80`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:47:02,591::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:47:02,591::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:47:02,592::task::1185::TaskManager.Task::(prepare) Task=`7d878a6b-7997-4cca-97d2-83f31f4d0e80`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:47:02,592::task::595::TaskManager.Task::(_updateState) Task=`7d878a6b-7997-4cca-97d2-83f31f4d0e80`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:47:02,592::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:47:02,592::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:47:02,592::task::990::TaskManager.Task::(_decref) Task=`7d878a6b-7997-4cca-97d2-83f31f4d0e80`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:47:02,597::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-338::DEBUG::2014-05-13 11:47:02,606::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6602059] >Thread-338::DEBUG::2014-05-13 11:47:02,607::task::595::TaskManager.Task::(_updateState) Task=`b373aa43-4fa9-4827-9118-b4a9f7fb1496`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:47:02,607::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:47:02,607::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f342128c-7cc4-40ba-a9a4-a4339ea054a1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:47:02,608::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:47:02,608::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:47:02,608::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f342128c-7cc4-40ba-a9a4-a4339ea054a1`::Granted request >Thread-338::DEBUG::2014-05-13 11:47:02,609::task::827::TaskManager.Task::(resourceAcquired) Task=`b373aa43-4fa9-4827-9118-b4a9f7fb1496`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:47:02,609::task::990::TaskManager.Task::(_decref) Task=`b373aa43-4fa9-4827-9118-b4a9f7fb1496`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:47:02,610::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:47:02,610::task::1185::TaskManager.Task::(prepare) Task=`b373aa43-4fa9-4827-9118-b4a9f7fb1496`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:47:02,611::task::595::TaskManager.Task::(_updateState) Task=`b373aa43-4fa9-4827-9118-b4a9f7fb1496`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:47:02,611::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:47:02,611::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:47:02,611::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:47:02,612::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:47:02,612::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:47:02,612::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:47:02,612::task::990::TaskManager.Task::(_decref) Task=`b373aa43-4fa9-4827-9118-b4a9f7fb1496`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:47:02,679::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0395337 s, 25.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:47:04,692::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:47:04,780::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316557 s, 32.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:47:05,329::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:47:06,793::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:47:06,875::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0395938 s, 25.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:47:08,348::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:47:08,888::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 11:47:08,911::task::595::TaskManager.Task::(_updateState) Task=`8fa673cb-ca7e-4c46-b35e-660143ef27cb`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:47:08,913::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:47:08,914::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000735169', 'lastCheck': '8.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:47:08,915::task::1185::TaskManager.Task::(prepare) Task=`8fa673cb-ca7e-4c46-b35e-660143ef27cb`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000735169', 'lastCheck': '8.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:47:08,916::task::595::TaskManager.Task::(_updateState) Task=`8fa673cb-ca7e-4c46-b35e-660143ef27cb`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:47:08,916::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:47:08,917::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:47:08,918::task::990::TaskManager.Task::(_decref) Task=`8fa673cb-ca7e-4c46-b35e-660143ef27cb`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:47:08,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.066036 s, 15.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:47:10,434::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:47:10,442::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00076681 s, 5.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:47:11,011::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:47:11,105::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0503573 s, 20.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:47:11,367::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:47:12,634::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [28fd42b9] >Thread-13::DEBUG::2014-05-13 11:47:12,634::task::595::TaskManager.Task::(_updateState) Task=`03cdfd03-223e-42cf-b9d9-06ad7f3e630e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:47:12,635::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:47:12,635::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:47:12,635::task::1185::TaskManager.Task::(prepare) Task=`03cdfd03-223e-42cf-b9d9-06ad7f3e630e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:47:12,635::task::595::TaskManager.Task::(_updateState) Task=`03cdfd03-223e-42cf-b9d9-06ad7f3e630e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:47:12,636::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:47:12,636::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:47:12,636::task::990::TaskManager.Task::(_decref) Task=`03cdfd03-223e-42cf-b9d9-06ad7f3e630e`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:47:12,641::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [28fd42b9] >Thread-338::DEBUG::2014-05-13 11:47:12,641::task::595::TaskManager.Task::(_updateState) Task=`1623058a-c3b5-4291-af44-b0ee3f107478`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:47:12,641::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:47:12,641::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`432acbde-85bd-43d5-b70e-233e04401337`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:47:12,642::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:47:12,642::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:47:12,642::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`432acbde-85bd-43d5-b70e-233e04401337`::Granted request >Thread-338::DEBUG::2014-05-13 11:47:12,642::task::827::TaskManager.Task::(resourceAcquired) Task=`1623058a-c3b5-4291-af44-b0ee3f107478`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:47:12,642::task::990::TaskManager.Task::(_decref) Task=`1623058a-c3b5-4291-af44-b0ee3f107478`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:47:12,643::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:47:12,643::task::1185::TaskManager.Task::(prepare) Task=`1623058a-c3b5-4291-af44-b0ee3f107478`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:47:12,643::task::595::TaskManager.Task::(_updateState) Task=`1623058a-c3b5-4291-af44-b0ee3f107478`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:47:12,644::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:47:12,644::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:47:12,644::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:47:12,644::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:47:12,644::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:47:12,644::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:47:12,645::task::990::TaskManager.Task::(_decref) Task=`1623058a-c3b5-4291-af44-b0ee3f107478`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:47:13,119::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:47:13,194::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318597 s, 32.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:47:14,385::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:47:15,207::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:47:15,284::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0335368 s, 30.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:47:17,297::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:47:17,377::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374341 s, 27.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:47:17,403::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:47:19,390::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:47:19,464::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320061 s, 32.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:47:20,421::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:47:20,449::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:47:20,457::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000606426 s, 6.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:47:21,477::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:47:21,552::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321667 s, 31.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:47:22,664::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3e537ec9] >Thread-13::DEBUG::2014-05-13 11:47:22,665::task::595::TaskManager.Task::(_updateState) Task=`ea765ecc-3da5-460e-b809-38600545bf4d`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:47:22,665::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:47:22,665::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:47:22,665::task::1185::TaskManager.Task::(prepare) Task=`ea765ecc-3da5-460e-b809-38600545bf4d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:47:22,666::task::595::TaskManager.Task::(_updateState) Task=`ea765ecc-3da5-460e-b809-38600545bf4d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:47:22,666::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:47:22,666::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:47:22,666::task::990::TaskManager.Task::(_decref) Task=`ea765ecc-3da5-460e-b809-38600545bf4d`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:47:22,672::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3e537ec9] >Thread-338::DEBUG::2014-05-13 11:47:22,672::task::595::TaskManager.Task::(_updateState) Task=`fd1c9156-be07-4b93-bb20-89c2063d96a9`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:47:22,672::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:47:22,673::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`32893bfb-66d8-4a37-afc3-548a33018b54`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:47:22,673::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:47:22,673::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:47:22,674::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`32893bfb-66d8-4a37-afc3-548a33018b54`::Granted request >Thread-338::DEBUG::2014-05-13 11:47:22,674::task::827::TaskManager.Task::(resourceAcquired) Task=`fd1c9156-be07-4b93-bb20-89c2063d96a9`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:47:22,674::task::990::TaskManager.Task::(_decref) Task=`fd1c9156-be07-4b93-bb20-89c2063d96a9`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:47:22,675::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:47:22,675::task::1185::TaskManager.Task::(prepare) Task=`fd1c9156-be07-4b93-bb20-89c2063d96a9`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:47:22,675::task::595::TaskManager.Task::(_updateState) Task=`fd1c9156-be07-4b93-bb20-89c2063d96a9`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:47:22,675::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:47:22,676::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:47:22,676::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:47:22,676::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:47:22,676::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:47:22,676::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:47:22,677::task::990::TaskManager.Task::(_decref) Task=`fd1c9156-be07-4b93-bb20-89c2063d96a9`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:47:23,439::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:47:23,565::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:47:23,646::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379804 s, 27.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:47:24,070::task::595::TaskManager.Task::(_updateState) Task=`43eb7c03-ef35-4763-b862-29e336dae905`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:47:24,070::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:47:24,070::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000606426', 'lastCheck': '3.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:47:24,070::task::1185::TaskManager.Task::(prepare) Task=`43eb7c03-ef35-4763-b862-29e336dae905`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000606426', 'lastCheck': '3.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:47:24,070::task::595::TaskManager.Task::(_updateState) Task=`43eb7c03-ef35-4763-b862-29e336dae905`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:47:24,071::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:47:24,071::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:47:24,071::task::990::TaskManager.Task::(_decref) Task=`43eb7c03-ef35-4763-b862-29e336dae905`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:47:25,659::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:47:25,753::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0511088 s, 20.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:47:26,458::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:47:27,766::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:47:27,850::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0405691 s, 25.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:47:29,489::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:47:29,863::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:47:29,944::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0376931 s, 27.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:47:30,464::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:47:30,471::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000759089 s, 5.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:47:31,957::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:47:32,035::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355026 s, 28.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:47:32,508::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:47:32,697::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:47:32,698::task::595::TaskManager.Task::(_updateState) Task=`cf3fd44b-92c0-422d-8bd0-2243a7b3b4b9`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:47:32,698::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:47:32,698::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:47:32,698::task::1185::TaskManager.Task::(prepare) Task=`cf3fd44b-92c0-422d-8bd0-2243a7b3b4b9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:47:32,699::task::595::TaskManager.Task::(_updateState) Task=`cf3fd44b-92c0-422d-8bd0-2243a7b3b4b9`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:47:32,699::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:47:32,699::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:47:32,699::task::990::TaskManager.Task::(_decref) Task=`cf3fd44b-92c0-422d-8bd0-2243a7b3b4b9`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:47:32,704::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:47:32,704::task::595::TaskManager.Task::(_updateState) Task=`77d06c63-9194-427b-81c2-15bb7d8f9f64`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:47:32,704::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:47:32,705::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`57c7dba7-851d-47c9-9b0b-65cc8a269457`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:47:32,705::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:47:32,705::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:47:32,705::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`57c7dba7-851d-47c9-9b0b-65cc8a269457`::Granted request >Thread-338::DEBUG::2014-05-13 11:47:32,706::task::827::TaskManager.Task::(resourceAcquired) Task=`77d06c63-9194-427b-81c2-15bb7d8f9f64`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:47:32,706::task::990::TaskManager.Task::(_decref) Task=`77d06c63-9194-427b-81c2-15bb7d8f9f64`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:47:32,707::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:47:32,707::task::1185::TaskManager.Task::(prepare) Task=`77d06c63-9194-427b-81c2-15bb7d8f9f64`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:47:32,707::task::595::TaskManager.Task::(_updateState) Task=`77d06c63-9194-427b-81c2-15bb7d8f9f64`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:47:32,707::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:47:32,707::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:47:32,708::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:47:32,708::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:47:32,708::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:47:32,708::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:47:32,708::task::990::TaskManager.Task::(_decref) Task=`77d06c63-9194-427b-81c2-15bb7d8f9f64`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:47:34,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:47:34,123::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314663 s, 32.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:47:35,526::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:47:36,136::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:47:36,222::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373129 s, 27.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:47:38,235::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:47:38,544::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:47:38,649::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.370777 s, 2.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:47:39,225::task::595::TaskManager.Task::(_updateState) Task=`296475b8-70cc-47b9-89cb-7fd411599343`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:47:39,250::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:47:39,250::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000759089', 'lastCheck': '8.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:47:39,250::task::1185::TaskManager.Task::(prepare) Task=`296475b8-70cc-47b9-89cb-7fd411599343`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000759089', 'lastCheck': '8.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:47:39,251::task::595::TaskManager.Task::(_updateState) Task=`296475b8-70cc-47b9-89cb-7fd411599343`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:47:39,251::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:47:39,251::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:47:39,251::task::990::TaskManager.Task::(_decref) Task=`296475b8-70cc-47b9-89cb-7fd411599343`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 11:47:40,478::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:47:40,486::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000750655 s, 5.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:47:40,662::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:47:40,740::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356253 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:47:41,562::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:47:42,729::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:47:42,729::task::595::TaskManager.Task::(_updateState) Task=`b7912e91-9cc8-4ec0-9a10-f072630127f4`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:47:42,730::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:47:42,730::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:47:42,730::task::1185::TaskManager.Task::(prepare) Task=`b7912e91-9cc8-4ec0-9a10-f072630127f4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:47:42,730::task::595::TaskManager.Task::(_updateState) Task=`b7912e91-9cc8-4ec0-9a10-f072630127f4`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:47:42,730::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:47:42,731::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:47:42,731::task::990::TaskManager.Task::(_decref) Task=`b7912e91-9cc8-4ec0-9a10-f072630127f4`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:47:42,735::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:47:42,735::task::595::TaskManager.Task::(_updateState) Task=`5c421ffd-f45b-4f7d-aeef-d015794cd03f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:47:42,736::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:47:42,736::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d8d3d348-f958-48b6-9979-eebbda3b71bb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:47:42,736::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:47:42,737::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:47:42,737::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d8d3d348-f958-48b6-9979-eebbda3b71bb`::Granted request >Thread-338::DEBUG::2014-05-13 11:47:42,738::task::827::TaskManager.Task::(resourceAcquired) Task=`5c421ffd-f45b-4f7d-aeef-d015794cd03f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:47:42,738::task::990::TaskManager.Task::(_decref) Task=`5c421ffd-f45b-4f7d-aeef-d015794cd03f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:47:42,739::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:47:42,739::task::1185::TaskManager.Task::(prepare) Task=`5c421ffd-f45b-4f7d-aeef-d015794cd03f`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:47:42,740::task::595::TaskManager.Task::(_updateState) Task=`5c421ffd-f45b-4f7d-aeef-d015794cd03f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:47:42,740::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:47:42,740::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:47:42,740::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:47:42,740::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:47:42,741::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:47:42,741::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:47:42,741::task::990::TaskManager.Task::(_decref) Task=`5c421ffd-f45b-4f7d-aeef-d015794cd03f`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:47:42,754::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:47:42,834::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384509 s, 26.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:47:44,595::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:47:44,847::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:47:44,928::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0368001 s, 27.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:47:46,941::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:47:47,015::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309813 s, 33.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:47:47,611::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:47:49,028::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:47:49,432::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.360197 s, 2.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:47:50,493::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:47:50,501::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000686631 s, 6.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:47:50,632::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:47:51,445::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:47:51,522::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322583 s, 31.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:47:52,760::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1cb1b71a] >Thread-13::DEBUG::2014-05-13 11:47:52,761::task::595::TaskManager.Task::(_updateState) Task=`ad803614-8035-45a4-a696-28e0ad64eebb`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:47:52,761::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:47:52,762::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:47:52,762::task::1185::TaskManager.Task::(prepare) Task=`ad803614-8035-45a4-a696-28e0ad64eebb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:47:52,762::task::595::TaskManager.Task::(_updateState) Task=`ad803614-8035-45a4-a696-28e0ad64eebb`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:47:52,762::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:47:52,762::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:47:52,762::task::990::TaskManager.Task::(_decref) Task=`ad803614-8035-45a4-a696-28e0ad64eebb`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:47:52,767::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1cb1b71a] >Thread-338::DEBUG::2014-05-13 11:47:52,768::task::595::TaskManager.Task::(_updateState) Task=`bf8e6738-da3f-43dc-a783-b5ee7a61604a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:47:52,768::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:47:52,768::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fe6bd1d8-cfb7-49d9-8c4e-b798c7a1b5bf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:47:52,768::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:47:52,769::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:47:52,769::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fe6bd1d8-cfb7-49d9-8c4e-b798c7a1b5bf`::Granted request >Thread-338::DEBUG::2014-05-13 11:47:52,769::task::827::TaskManager.Task::(resourceAcquired) Task=`bf8e6738-da3f-43dc-a783-b5ee7a61604a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:47:52,769::task::990::TaskManager.Task::(_decref) Task=`bf8e6738-da3f-43dc-a783-b5ee7a61604a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:47:52,770::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:47:52,770::task::1185::TaskManager.Task::(prepare) Task=`bf8e6738-da3f-43dc-a783-b5ee7a61604a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:47:52,771::task::595::TaskManager.Task::(_updateState) Task=`bf8e6738-da3f-43dc-a783-b5ee7a61604a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:47:52,771::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:47:52,771::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:47:52,771::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:47:52,771::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:47:52,772::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:47:52,772::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:47:52,772::task::990::TaskManager.Task::(_decref) Task=`bf8e6738-da3f-43dc-a783-b5ee7a61604a`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:47:53,534::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:47:53,615::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0369221 s, 27.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:47:53,650::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:47:54,665::task::595::TaskManager.Task::(_updateState) Task=`df8a0ae2-6cf5-4e00-81d9-28ce36b99721`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:47:54,665::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:47:54,665::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000686631', 'lastCheck': '4.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:47:54,665::task::1185::TaskManager.Task::(prepare) Task=`df8a0ae2-6cf5-4e00-81d9-28ce36b99721`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000686631', 'lastCheck': '4.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:47:54,665::task::595::TaskManager.Task::(_updateState) Task=`df8a0ae2-6cf5-4e00-81d9-28ce36b99721`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:47:54,665::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:47:54,666::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:47:54,666::task::990::TaskManager.Task::(_decref) Task=`df8a0ae2-6cf5-4e00-81d9-28ce36b99721`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:47:55,628::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:47:55,701::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316536 s, 32.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:47:56,665::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:47:57,715::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:47:57,811::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0403135 s, 25.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:47:59,683::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:47:59,824::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:47:59,906::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0377906 s, 27.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:48:00,508::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:48:00,516::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000623337 s, 6.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:48:01,919::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:48:02,001::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0388926 s, 26.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:48:02,708::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:48:02,790::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:48:02,791::task::595::TaskManager.Task::(_updateState) Task=`2d73b8ff-61ac-4b51-9d6c-32ba3e122566`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:48:02,791::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:48:02,791::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:48:02,792::task::1185::TaskManager.Task::(prepare) Task=`2d73b8ff-61ac-4b51-9d6c-32ba3e122566`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:48:02,792::task::595::TaskManager.Task::(_updateState) Task=`2d73b8ff-61ac-4b51-9d6c-32ba3e122566`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:48:02,792::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:48:02,792::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:48:02,792::task::990::TaskManager.Task::(_decref) Task=`2d73b8ff-61ac-4b51-9d6c-32ba3e122566`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:48:02,797::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:48:02,797::task::595::TaskManager.Task::(_updateState) Task=`49c22d51-331d-400b-b97d-801425f3aa82`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:48:02,797::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:48:02,798::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6a660755-0796-434f-bd8b-c8df98cc85aa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:48:02,798::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:48:02,798::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:48:02,798::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6a660755-0796-434f-bd8b-c8df98cc85aa`::Granted request >Thread-338::DEBUG::2014-05-13 11:48:02,799::task::827::TaskManager.Task::(resourceAcquired) Task=`49c22d51-331d-400b-b97d-801425f3aa82`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:48:02,799::task::990::TaskManager.Task::(_decref) Task=`49c22d51-331d-400b-b97d-801425f3aa82`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:48:02,799::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:48:02,800::task::1185::TaskManager.Task::(prepare) Task=`49c22d51-331d-400b-b97d-801425f3aa82`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:48:02,800::task::595::TaskManager.Task::(_updateState) Task=`49c22d51-331d-400b-b97d-801425f3aa82`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:48:02,800::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:48:02,800::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:48:02,800::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:48:02,801::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:48:02,801::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:48:02,801::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:48:02,801::task::990::TaskManager.Task::(_decref) Task=`49c22d51-331d-400b-b97d-801425f3aa82`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:48:04,014::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:48:04,383::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.326321 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:48:05,729::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:48:06,396::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:48:06,470::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031047 s, 33.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:48:08,483::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:48:08,562::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358307 s, 28.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:48:08,747::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:48:09,804::task::595::TaskManager.Task::(_updateState) Task=`16133315-b16d-42e9-83da-eac53c5e7f4a`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:48:09,804::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:48:09,804::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000623337', 'lastCheck': '9.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:48:09,804::task::1185::TaskManager.Task::(prepare) Task=`16133315-b16d-42e9-83da-eac53c5e7f4a`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000623337', 'lastCheck': '9.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:48:09,805::task::595::TaskManager.Task::(_updateState) Task=`16133315-b16d-42e9-83da-eac53c5e7f4a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:48:09,805::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:48:09,805::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:48:09,805::task::990::TaskManager.Task::(_decref) Task=`16133315-b16d-42e9-83da-eac53c5e7f4a`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 11:48:10,523::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:48:10,531::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000583498 s, 7.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:48:10,575::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:48:10,653::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352815 s, 29.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:48:11,765::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:48:12,666::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:48:12,744::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356621 s, 28.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:48:12,821::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:48:12,822::task::595::TaskManager.Task::(_updateState) Task=`2ae66588-d646-429b-a30c-e918aff3f3d2`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:48:12,822::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:48:12,822::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:48:12,822::task::1185::TaskManager.Task::(prepare) Task=`2ae66588-d646-429b-a30c-e918aff3f3d2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:48:12,823::task::595::TaskManager.Task::(_updateState) Task=`2ae66588-d646-429b-a30c-e918aff3f3d2`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:48:12,823::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:48:12,823::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:48:12,823::task::990::TaskManager.Task::(_decref) Task=`2ae66588-d646-429b-a30c-e918aff3f3d2`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:48:12,828::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:48:12,829::task::595::TaskManager.Task::(_updateState) Task=`f3b92993-a09c-4da4-aa3c-34c5079257d3`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:48:12,829::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:48:12,829::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f07a4fc3-482f-4cea-85ce-fc319939407a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:48:12,830::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:48:12,830::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:48:12,830::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f07a4fc3-482f-4cea-85ce-fc319939407a`::Granted request >Thread-338::DEBUG::2014-05-13 11:48:12,830::task::827::TaskManager.Task::(resourceAcquired) Task=`f3b92993-a09c-4da4-aa3c-34c5079257d3`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:48:12,831::task::990::TaskManager.Task::(_decref) Task=`f3b92993-a09c-4da4-aa3c-34c5079257d3`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:48:12,831::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:48:12,832::task::1185::TaskManager.Task::(prepare) Task=`f3b92993-a09c-4da4-aa3c-34c5079257d3`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:48:12,832::task::595::TaskManager.Task::(_updateState) Task=`f3b92993-a09c-4da4-aa3c-34c5079257d3`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:48:12,832::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:48:12,832::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:48:12,832::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:48:12,833::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:48:12,833::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:48:12,833::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:48:12,833::task::990::TaskManager.Task::(_decref) Task=`f3b92993-a09c-4da4-aa3c-34c5079257d3`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:48:14,757::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:48:14,783::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:48:15,166::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.366255 s, 2.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:48:17,179::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:48:17,254::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317826 s, 32.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:48:17,806::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:48:19,268::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:48:19,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320532 s, 31.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:48:20,538::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:48:20,546::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000777715 s, 5.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:48:20,829::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:48:21,357::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:48:21,431::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031543 s, 32.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:48:22,853::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:48:22,854::task::595::TaskManager.Task::(_updateState) Task=`7492af03-387a-43c3-8790-214567379603`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:48:22,854::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:48:22,854::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:48:22,854::task::1185::TaskManager.Task::(prepare) Task=`7492af03-387a-43c3-8790-214567379603`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:48:22,855::task::595::TaskManager.Task::(_updateState) Task=`7492af03-387a-43c3-8790-214567379603`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:48:22,855::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:48:22,855::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:48:22,855::task::990::TaskManager.Task::(_decref) Task=`7492af03-387a-43c3-8790-214567379603`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:48:22,860::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:48:22,860::task::595::TaskManager.Task::(_updateState) Task=`e315eb1d-49c3-4761-9056-8e8d174c7561`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:48:22,861::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:48:22,861::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`21c752a9-99f1-4bf8-8ed2-deb4da5c1e6b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:48:22,861::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:48:22,862::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:48:22,862::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`21c752a9-99f1-4bf8-8ed2-deb4da5c1e6b`::Granted request >Thread-338::DEBUG::2014-05-13 11:48:22,862::task::827::TaskManager.Task::(resourceAcquired) Task=`e315eb1d-49c3-4761-9056-8e8d174c7561`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:48:22,862::task::990::TaskManager.Task::(_decref) Task=`e315eb1d-49c3-4761-9056-8e8d174c7561`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:48:22,863::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:48:22,863::task::1185::TaskManager.Task::(prepare) Task=`e315eb1d-49c3-4761-9056-8e8d174c7561`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:48:22,863::task::595::TaskManager.Task::(_updateState) Task=`e315eb1d-49c3-4761-9056-8e8d174c7561`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:48:22,864::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:48:22,864::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:48:22,864::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:48:22,864::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:48:22,864::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:48:22,864::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:48:22,865::task::990::TaskManager.Task::(_decref) Task=`e315eb1d-49c3-4761-9056-8e8d174c7561`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:48:23,444::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:48:23,522::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352263 s, 29.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:48:23,848::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:48:24,939::task::595::TaskManager.Task::(_updateState) Task=`60f0c590-46f9-4c9e-bbc8-658103c56fec`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:48:24,939::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:48:24,939::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000777715', 'lastCheck': '4.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:48:24,939::task::1185::TaskManager.Task::(prepare) Task=`60f0c590-46f9-4c9e-bbc8-658103c56fec`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000777715', 'lastCheck': '4.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:48:24,939::task::595::TaskManager.Task::(_updateState) Task=`60f0c590-46f9-4c9e-bbc8-658103c56fec`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:48:24,939::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:48:24,940::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:48:24,940::task::990::TaskManager.Task::(_decref) Task=`60f0c590-46f9-4c9e-bbc8-658103c56fec`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:48:25,535::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:48:25,611::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320848 s, 31.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:48:26,868::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:48:27,625::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:48:27,703::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359747 s, 28.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:48:29,716::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:48:29,887::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:48:30,091::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.332523 s, 3.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:48:30,553::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:48:30,560::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000684898 s, 6.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:48:32,104::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:48:32,185::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0392911 s, 26.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:48:32,884::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17998964] >Thread-13::DEBUG::2014-05-13 11:48:32,885::task::595::TaskManager.Task::(_updateState) Task=`ab209e56-f78a-4299-82d8-c8d9f312cd78`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:48:32,885::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:48:32,886::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:48:32,886::task::1185::TaskManager.Task::(prepare) Task=`ab209e56-f78a-4299-82d8-c8d9f312cd78`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:48:32,886::task::595::TaskManager.Task::(_updateState) Task=`ab209e56-f78a-4299-82d8-c8d9f312cd78`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:48:32,886::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:48:32,886::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:48:32,887::task::990::TaskManager.Task::(_decref) Task=`ab209e56-f78a-4299-82d8-c8d9f312cd78`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:48:32,891::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17998964] >Thread-338::DEBUG::2014-05-13 11:48:32,892::task::595::TaskManager.Task::(_updateState) Task=`32c76522-9c79-4ac0-9459-63416a26fe7f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:48:32,892::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:48:32,892::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4eddcc34-ec1c-4870-817b-ba9c24802c90`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:48:32,893::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:48:32,893::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:48:32,893::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4eddcc34-ec1c-4870-817b-ba9c24802c90`::Granted request >Thread-338::DEBUG::2014-05-13 11:48:32,893::task::827::TaskManager.Task::(resourceAcquired) Task=`32c76522-9c79-4ac0-9459-63416a26fe7f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:48:32,894::task::990::TaskManager.Task::(_decref) Task=`32c76522-9c79-4ac0-9459-63416a26fe7f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:48:32,894::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:48:32,895::task::1185::TaskManager.Task::(prepare) Task=`32c76522-9c79-4ac0-9459-63416a26fe7f`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:48:32,895::task::595::TaskManager.Task::(_updateState) Task=`32c76522-9c79-4ac0-9459-63416a26fe7f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:48:32,895::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:48:32,895::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:48:32,895::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:48:32,896::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:48:32,896::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:48:32,896::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:48:32,896::task::990::TaskManager.Task::(_decref) Task=`32c76522-9c79-4ac0-9459-63416a26fe7f`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:48:32,906::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:48:34,198::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:48:34,273::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318719 s, 32.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:48:35,924::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:48:36,286::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:48:36,364::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0336652 s, 30.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:48:38,377::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:48:38,457::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0369352 s, 27.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:48:38,945::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:48:40,069::task::595::TaskManager.Task::(_updateState) Task=`b63d6192-c93f-429f-9906-f71a07416de5`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:48:40,070::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:48:40,070::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000684898', 'lastCheck': '9.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:48:40,070::task::1185::TaskManager.Task::(prepare) Task=`b63d6192-c93f-429f-9906-f71a07416de5`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000684898', 'lastCheck': '9.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:48:40,070::task::595::TaskManager.Task::(_updateState) Task=`b63d6192-c93f-429f-9906-f71a07416de5`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:48:40,070::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:48:40,070::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:48:40,071::task::990::TaskManager.Task::(_decref) Task=`b63d6192-c93f-429f-9906-f71a07416de5`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:48:40,470::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:48:40,546::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318776 s, 32.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:48:40,568::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:48:40,576::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000704377 s, 5.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:48:41,965::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:48:42,559::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:48:42,652::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0511004 s, 20.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:48:42,918::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:48:42,918::task::595::TaskManager.Task::(_updateState) Task=`0723801a-532e-4b04-a516-5795aa8dc77a`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:48:42,918::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:48:42,919::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:48:42,919::task::1185::TaskManager.Task::(prepare) Task=`0723801a-532e-4b04-a516-5795aa8dc77a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:48:42,919::task::595::TaskManager.Task::(_updateState) Task=`0723801a-532e-4b04-a516-5795aa8dc77a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:48:42,919::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:48:42,919::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:48:42,920::task::990::TaskManager.Task::(_decref) Task=`0723801a-532e-4b04-a516-5795aa8dc77a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:48:42,924::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:48:42,924::task::595::TaskManager.Task::(_updateState) Task=`434068bd-86a0-4ee8-a4d1-1a2533f8604a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:48:42,925::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:48:42,925::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a502250b-6913-45e0-81ea-c36b15103f77`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:48:42,925::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:48:42,926::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:48:42,926::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a502250b-6913-45e0-81ea-c36b15103f77`::Granted request >Thread-338::DEBUG::2014-05-13 11:48:42,926::task::827::TaskManager.Task::(resourceAcquired) Task=`434068bd-86a0-4ee8-a4d1-1a2533f8604a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:48:42,926::task::990::TaskManager.Task::(_decref) Task=`434068bd-86a0-4ee8-a4d1-1a2533f8604a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:48:42,927::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:48:42,927::task::1185::TaskManager.Task::(prepare) Task=`434068bd-86a0-4ee8-a4d1-1a2533f8604a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:48:42,927::task::595::TaskManager.Task::(_updateState) Task=`434068bd-86a0-4ee8-a4d1-1a2533f8604a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:48:42,928::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:48:42,928::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:48:42,928::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:48:42,928::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:48:42,928::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:48:42,928::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:48:42,929::task::990::TaskManager.Task::(_decref) Task=`434068bd-86a0-4ee8-a4d1-1a2533f8604a`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:48:44,665::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:48:44,745::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0366708 s, 27.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:48:44,986::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:48:46,758::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:48:46,834::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0339371 s, 30.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:48:48,004::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:48:48,847::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:48:48,921::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316154 s, 32.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:48:50,583::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:48:50,592::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.0013784 s, 3.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:48:50,934::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:48:51,016::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379508 s, 27.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:48:51,023::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:48:52,947::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:48:52,948::task::595::TaskManager.Task::(_updateState) Task=`43e155bd-b621-477e-aad1-0d264e12ce94`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:48:52,948::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:48:52,949::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:48:52,949::task::1185::TaskManager.Task::(prepare) Task=`43e155bd-b621-477e-aad1-0d264e12ce94`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:48:52,949::task::595::TaskManager.Task::(_updateState) Task=`43e155bd-b621-477e-aad1-0d264e12ce94`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:48:52,949::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:48:52,949::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:48:52,949::task::990::TaskManager.Task::(_decref) Task=`43e155bd-b621-477e-aad1-0d264e12ce94`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:48:52,954::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:48:52,954::task::595::TaskManager.Task::(_updateState) Task=`6251c976-e764-4a58-a40e-e8ee947a4d3e`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:48:52,954::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:48:52,955::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2a2fa2bb-929d-4635-934b-c928ae8d5684`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:48:52,955::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:48:52,955::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:48:52,955::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2a2fa2bb-929d-4635-934b-c928ae8d5684`::Granted request >Thread-338::DEBUG::2014-05-13 11:48:52,957::task::827::TaskManager.Task::(resourceAcquired) Task=`6251c976-e764-4a58-a40e-e8ee947a4d3e`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:48:52,957::task::990::TaskManager.Task::(_decref) Task=`6251c976-e764-4a58-a40e-e8ee947a4d3e`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:48:52,958::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:48:52,958::task::1185::TaskManager.Task::(prepare) Task=`6251c976-e764-4a58-a40e-e8ee947a4d3e`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:48:52,958::task::595::TaskManager.Task::(_updateState) Task=`6251c976-e764-4a58-a40e-e8ee947a4d3e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:48:52,958::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:48:52,959::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:48:52,959::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:48:52,959::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:48:52,959::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:48:52,959::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:48:52,960::task::990::TaskManager.Task::(_decref) Task=`6251c976-e764-4a58-a40e-e8ee947a4d3e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:48:53,031::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:48:53,110::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360569 s, 28.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:48:54,042::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:48:55,123::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:48:55,199::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322391 s, 31.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:48:55,208::task::595::TaskManager.Task::(_updateState) Task=`731de27d-110f-483e-802a-c96c255d58c4`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:48:55,209::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:48:55,210::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0013784', 'lastCheck': '4.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:48:55,211::task::1185::TaskManager.Task::(prepare) Task=`731de27d-110f-483e-802a-c96c255d58c4`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0013784', 'lastCheck': '4.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:48:55,211::task::595::TaskManager.Task::(_updateState) Task=`731de27d-110f-483e-802a-c96c255d58c4`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:48:55,212::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:48:55,213::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:48:55,214::task::990::TaskManager.Task::(_decref) Task=`731de27d-110f-483e-802a-c96c255d58c4`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:48:57,063::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:48:57,233::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:48:57,330::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0529965 s, 19.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:48:59,342::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:48:59,424::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387115 s, 26.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:49:00,082::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:49:00,598::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:49:00,606::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000753824 s, 5.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:49:01,437::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:01,517::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0369838 s, 27.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:49:02,978::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:49:02,978::task::595::TaskManager.Task::(_updateState) Task=`76d4e9e2-b6c0-492e-88f6-bbb1da28b7e4`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:49:02,979::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:49:02,979::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:49:02,979::task::1185::TaskManager.Task::(prepare) Task=`76d4e9e2-b6c0-492e-88f6-bbb1da28b7e4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:49:02,979::task::595::TaskManager.Task::(_updateState) Task=`76d4e9e2-b6c0-492e-88f6-bbb1da28b7e4`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:49:02,979::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:49:02,980::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:49:02,980::task::990::TaskManager.Task::(_decref) Task=`76d4e9e2-b6c0-492e-88f6-bbb1da28b7e4`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:49:02,985::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:49:02,985::task::595::TaskManager.Task::(_updateState) Task=`d765728a-ab5c-4910-a601-97dceeab80ee`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:49:02,985::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:49:02,986::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ee7b655e-e06c-4281-97db-c104351078b7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:49:02,986::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:49:02,986::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:49:02,986::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ee7b655e-e06c-4281-97db-c104351078b7`::Granted request >Thread-338::DEBUG::2014-05-13 11:49:02,987::task::827::TaskManager.Task::(resourceAcquired) Task=`d765728a-ab5c-4910-a601-97dceeab80ee`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:49:02,987::task::990::TaskManager.Task::(_decref) Task=`d765728a-ab5c-4910-a601-97dceeab80ee`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:49:02,988::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:49:02,988::task::1185::TaskManager.Task::(prepare) Task=`d765728a-ab5c-4910-a601-97dceeab80ee`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:49:02,988::task::595::TaskManager.Task::(_updateState) Task=`d765728a-ab5c-4910-a601-97dceeab80ee`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:49:02,988::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:49:02,988::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:49:02,988::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:49:02,989::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:49:02,989::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:49:02,989::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:49:02,989::task::990::TaskManager.Task::(_decref) Task=`d765728a-ab5c-4910-a601-97dceeab80ee`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:49:03,104::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:49:03,530::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:03,606::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0339549 s, 30.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:49:05,620::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:05,746::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0823797 s, 12.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:49:06,122::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:49:07,760::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:07,841::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0378702 s, 27.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:49:09,144::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:49:09,854::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:09,929::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314198 s, 32.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:49:10,355::task::595::TaskManager.Task::(_updateState) Task=`2a47164d-fcc2-44cd-a9f9-9b5e74abe8b7`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:49:10,355::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:49:10,355::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000753824', 'lastCheck': '9.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:49:10,356::task::1185::TaskManager.Task::(prepare) Task=`2a47164d-fcc2-44cd-a9f9-9b5e74abe8b7`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000753824', 'lastCheck': '9.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:49:10,356::task::595::TaskManager.Task::(_updateState) Task=`2a47164d-fcc2-44cd-a9f9-9b5e74abe8b7`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:49:10,356::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:49:10,356::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:49:10,356::task::990::TaskManager.Task::(_decref) Task=`2a47164d-fcc2-44cd-a9f9-9b5e74abe8b7`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 11:49:10,613::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:49:10,622::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000752621 s, 5.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:49:11,942::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:12,036::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0512254 s, 20.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:49:12,179::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:49:13,008::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:49:13,008::task::595::TaskManager.Task::(_updateState) Task=`29ea46df-672c-4983-b413-4e9b9cf7bf53`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:49:13,008::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:49:13,009::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:49:13,009::task::1185::TaskManager.Task::(prepare) Task=`29ea46df-672c-4983-b413-4e9b9cf7bf53`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:49:13,009::task::595::TaskManager.Task::(_updateState) Task=`29ea46df-672c-4983-b413-4e9b9cf7bf53`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:49:13,009::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:49:13,010::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:49:13,010::task::990::TaskManager.Task::(_decref) Task=`29ea46df-672c-4983-b413-4e9b9cf7bf53`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:49:13,014::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:49:13,014::task::595::TaskManager.Task::(_updateState) Task=`1d21468f-2bb0-43d1-87a5-7df436522ee3`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:49:13,015::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:49:13,015::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`63795fb6-41d8-4a6d-8396-88b0e5bce043`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:49:13,015::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:49:13,016::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:49:13,016::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`63795fb6-41d8-4a6d-8396-88b0e5bce043`::Granted request >Thread-338::DEBUG::2014-05-13 11:49:13,016::task::827::TaskManager.Task::(resourceAcquired) Task=`1d21468f-2bb0-43d1-87a5-7df436522ee3`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:49:13,016::task::990::TaskManager.Task::(_decref) Task=`1d21468f-2bb0-43d1-87a5-7df436522ee3`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:49:13,017::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:49:13,017::task::1185::TaskManager.Task::(prepare) Task=`1d21468f-2bb0-43d1-87a5-7df436522ee3`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:49:13,017::task::595::TaskManager.Task::(_updateState) Task=`1d21468f-2bb0-43d1-87a5-7df436522ee3`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:49:13,018::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:49:13,018::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:49:13,018::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:49:13,018::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:49:13,018::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:49:13,018::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:49:13,019::task::990::TaskManager.Task::(_decref) Task=`1d21468f-2bb0-43d1-87a5-7df436522ee3`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:49:14,049::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:14,129::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036406 s, 28.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:49:15,198::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:49:16,142::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:16,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.032426 s, 31.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:49:18,223::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:49:18,231::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:18,310::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364954 s, 28.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:49:20,323::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:20,400::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323085 s, 31.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:49:20,629::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:49:20,637::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000835272 s, 4.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:49:21,242::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:49:22,413::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:22,492::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321144 s, 31.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:49:23,038::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:49:23,039::task::595::TaskManager.Task::(_updateState) Task=`c2ba594f-9ece-4765-9de3-ae93bd3188e7`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:49:23,039::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:49:23,039::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:49:23,039::task::1185::TaskManager.Task::(prepare) Task=`c2ba594f-9ece-4765-9de3-ae93bd3188e7`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:49:23,040::task::595::TaskManager.Task::(_updateState) Task=`c2ba594f-9ece-4765-9de3-ae93bd3188e7`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:49:23,040::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:49:23,040::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:49:23,040::task::990::TaskManager.Task::(_decref) Task=`c2ba594f-9ece-4765-9de3-ae93bd3188e7`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:49:23,045::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:49:23,045::task::595::TaskManager.Task::(_updateState) Task=`8025a5f0-18a8-45d0-8ac9-bd7450af25be`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:49:23,046::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:49:23,046::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6e8cc9af-e79f-4103-9290-bacf603e0952`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:49:23,046::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:49:23,047::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:49:23,047::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6e8cc9af-e79f-4103-9290-bacf603e0952`::Granted request >Thread-338::DEBUG::2014-05-13 11:49:23,047::task::827::TaskManager.Task::(resourceAcquired) Task=`8025a5f0-18a8-45d0-8ac9-bd7450af25be`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:49:23,047::task::990::TaskManager.Task::(_decref) Task=`8025a5f0-18a8-45d0-8ac9-bd7450af25be`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:49:23,048::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:49:23,048::task::1185::TaskManager.Task::(prepare) Task=`8025a5f0-18a8-45d0-8ac9-bd7450af25be`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:49:23,048::task::595::TaskManager.Task::(_updateState) Task=`8025a5f0-18a8-45d0-8ac9-bd7450af25be`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:49:23,049::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:49:23,049::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:49:23,049::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:49:23,049::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:49:23,049::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:49:23,049::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:49:23,050::task::990::TaskManager.Task::(_decref) Task=`8025a5f0-18a8-45d0-8ac9-bd7450af25be`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:49:24,261::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:49:24,505::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:24,581::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317338 s, 32.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:49:25,488::task::595::TaskManager.Task::(_updateState) Task=`000d32dc-5126-4ab5-96ef-efc925beba12`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:49:25,489::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:49:25,489::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000835272', 'lastCheck': '4.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:49:25,489::task::1185::TaskManager.Task::(prepare) Task=`000d32dc-5126-4ab5-96ef-efc925beba12`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000835272', 'lastCheck': '4.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:49:25,489::task::595::TaskManager.Task::(_updateState) Task=`000d32dc-5126-4ab5-96ef-efc925beba12`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:49:25,489::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:49:25,489::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:49:25,490::task::990::TaskManager.Task::(_decref) Task=`000d32dc-5126-4ab5-96ef-efc925beba12`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:49:26,594::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:26,687::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0503709 s, 20.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:49:27,280::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:49:28,700::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:28,780::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379089 s, 27.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:49:30,298::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:49:30,644::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:49:30,652::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000693682 s, 5.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:49:30,793::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:30,874::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365112 s, 28.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:49:32,887::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:32,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360424 s, 28.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:49:33,068::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:49:33,068::task::595::TaskManager.Task::(_updateState) Task=`bea18df9-4a2a-4ed0-adee-ca2d675314a1`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:49:33,068::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:49:33,069::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:49:33,069::task::1185::TaskManager.Task::(prepare) Task=`bea18df9-4a2a-4ed0-adee-ca2d675314a1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:49:33,069::task::595::TaskManager.Task::(_updateState) Task=`bea18df9-4a2a-4ed0-adee-ca2d675314a1`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:49:33,069::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:49:33,069::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:49:33,070::task::990::TaskManager.Task::(_decref) Task=`bea18df9-4a2a-4ed0-adee-ca2d675314a1`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:49:33,074::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:49:33,075::task::595::TaskManager.Task::(_updateState) Task=`8ce2c75e-a329-47ac-8275-34699ed2c8cb`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:49:33,075::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:49:33,075::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`322c99ab-8fa0-4648-ad16-cae1d62f814e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:49:33,075::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:49:33,076::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:49:33,076::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`322c99ab-8fa0-4648-ad16-cae1d62f814e`::Granted request >Thread-338::DEBUG::2014-05-13 11:49:33,076::task::827::TaskManager.Task::(resourceAcquired) Task=`8ce2c75e-a329-47ac-8275-34699ed2c8cb`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:49:33,076::task::990::TaskManager.Task::(_decref) Task=`8ce2c75e-a329-47ac-8275-34699ed2c8cb`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:49:33,077::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:49:33,077::task::1185::TaskManager.Task::(prepare) Task=`8ce2c75e-a329-47ac-8275-34699ed2c8cb`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:49:33,077::task::595::TaskManager.Task::(_updateState) Task=`8ce2c75e-a329-47ac-8275-34699ed2c8cb`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:49:33,078::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:49:33,078::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:49:33,078::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:49:33,078::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:49:33,078::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:49:33,079::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:49:33,079::task::990::TaskManager.Task::(_decref) Task=`8ce2c75e-a329-47ac-8275-34699ed2c8cb`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:49:33,318::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:49:34,980::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:35,054::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031305 s, 32.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:49:36,338::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:49:37,066::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:37,141::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309546 s, 33.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:49:39,154::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:39,234::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0367454 s, 27.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:49:39,355::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:49:40,625::task::595::TaskManager.Task::(_updateState) Task=`945ebd64-0e22-4b48-b479-f69e2c3d083c`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:49:40,625::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:49:40,625::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000693682', 'lastCheck': '10.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:49:40,626::task::1185::TaskManager.Task::(prepare) Task=`945ebd64-0e22-4b48-b479-f69e2c3d083c`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000693682', 'lastCheck': '10.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:49:40,626::task::595::TaskManager.Task::(_updateState) Task=`945ebd64-0e22-4b48-b479-f69e2c3d083c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:49:40,626::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:49:40,626::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:49:40,626::task::990::TaskManager.Task::(_decref) Task=`945ebd64-0e22-4b48-b479-f69e2c3d083c`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 11:49:40,658::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:49:40,666::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000699209 s, 5.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:49:41,247::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:41,339::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0496879 s, 20.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:49:42,375::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:49:43,098::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6849800f] >Thread-13::DEBUG::2014-05-13 11:49:43,099::task::595::TaskManager.Task::(_updateState) Task=`beb19782-a77b-480c-ba9c-b14205e52070`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:49:43,099::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:49:43,100::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:49:43,100::task::1185::TaskManager.Task::(prepare) Task=`beb19782-a77b-480c-ba9c-b14205e52070`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:49:43,100::task::595::TaskManager.Task::(_updateState) Task=`beb19782-a77b-480c-ba9c-b14205e52070`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:49:43,100::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:49:43,100::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:49:43,100::task::990::TaskManager.Task::(_decref) Task=`beb19782-a77b-480c-ba9c-b14205e52070`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:49:43,105::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6849800f] >Thread-338::DEBUG::2014-05-13 11:49:43,105::task::595::TaskManager.Task::(_updateState) Task=`c5cceb5e-2f52-4887-adbb-7afd8bebccf0`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:49:43,106::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:49:43,106::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2417c3e1-f8c6-4ad7-9b4d-e54c2bb17e17`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:49:43,106::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:49:43,107::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:49:43,107::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2417c3e1-f8c6-4ad7-9b4d-e54c2bb17e17`::Granted request >Thread-338::DEBUG::2014-05-13 11:49:43,107::task::827::TaskManager.Task::(resourceAcquired) Task=`c5cceb5e-2f52-4887-adbb-7afd8bebccf0`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:49:43,107::task::990::TaskManager.Task::(_decref) Task=`c5cceb5e-2f52-4887-adbb-7afd8bebccf0`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:49:43,108::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:49:43,108::task::1185::TaskManager.Task::(prepare) Task=`c5cceb5e-2f52-4887-adbb-7afd8bebccf0`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:49:43,108::task::595::TaskManager.Task::(_updateState) Task=`c5cceb5e-2f52-4887-adbb-7afd8bebccf0`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:49:43,109::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:49:43,109::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:49:43,109::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:49:43,109::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:49:43,109::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:49:43,110::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:49:43,110::task::990::TaskManager.Task::(_decref) Task=`c5cceb5e-2f52-4887-adbb-7afd8bebccf0`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:49:43,351::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:43,431::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036021 s, 28.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:49:45,393::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:49:45,444::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:45,520::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318638 s, 32.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:49:47,534::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:47,614::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037277 s, 27.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:49:48,411::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:49:49,627::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:49,701::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311086 s, 32.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:49:50,674::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:49:50,681::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000717783 s, 5.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:49:51,431::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:49:51,714::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:51,788::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314268 s, 32.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:49:53,128::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:49:53,129::task::595::TaskManager.Task::(_updateState) Task=`3460136e-5c29-4e01-a7e7-9aa81dd01028`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:49:53,129::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:49:53,129::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:49:53,130::task::1185::TaskManager.Task::(prepare) Task=`3460136e-5c29-4e01-a7e7-9aa81dd01028`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:49:53,130::task::595::TaskManager.Task::(_updateState) Task=`3460136e-5c29-4e01-a7e7-9aa81dd01028`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:49:53,130::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:49:53,130::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:49:53,130::task::990::TaskManager.Task::(_decref) Task=`3460136e-5c29-4e01-a7e7-9aa81dd01028`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:49:53,134::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:49:53,135::task::595::TaskManager.Task::(_updateState) Task=`ea33bc1a-1dd7-4b65-ac93-5e5efcdc3f59`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:49:53,135::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:49:53,135::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`03adb7c5-8a8e-404f-b56e-2a65960e8681`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:49:53,136::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:49:53,136::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:49:53,136::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`03adb7c5-8a8e-404f-b56e-2a65960e8681`::Granted request >Thread-338::DEBUG::2014-05-13 11:49:53,136::task::827::TaskManager.Task::(resourceAcquired) Task=`ea33bc1a-1dd7-4b65-ac93-5e5efcdc3f59`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:49:53,136::task::990::TaskManager.Task::(_decref) Task=`ea33bc1a-1dd7-4b65-ac93-5e5efcdc3f59`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:49:53,137::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:49:53,137::task::1185::TaskManager.Task::(prepare) Task=`ea33bc1a-1dd7-4b65-ac93-5e5efcdc3f59`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:49:53,138::task::595::TaskManager.Task::(_updateState) Task=`ea33bc1a-1dd7-4b65-ac93-5e5efcdc3f59`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:49:53,138::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:49:53,138::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:49:53,138::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:49:53,138::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:49:53,139::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:49:53,139::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:49:53,139::task::990::TaskManager.Task::(_decref) Task=`ea33bc1a-1dd7-4b65-ac93-5e5efcdc3f59`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:49:53,801::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:53,895::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0508624 s, 20.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:49:54,451::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:49:55,762::task::595::TaskManager.Task::(_updateState) Task=`58013a8d-369d-42bd-ae7a-b4896dc4cb2e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:49:55,762::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:49:55,762::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000717783', 'lastCheck': '5.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:49:55,762::task::1185::TaskManager.Task::(prepare) Task=`58013a8d-369d-42bd-ae7a-b4896dc4cb2e`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000717783', 'lastCheck': '5.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:49:55,762::task::595::TaskManager.Task::(_updateState) Task=`58013a8d-369d-42bd-ae7a-b4896dc4cb2e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:49:55,762::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:49:55,763::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:49:55,763::task::990::TaskManager.Task::(_decref) Task=`58013a8d-369d-42bd-ae7a-b4896dc4cb2e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:49:55,908::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:55,983::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0330962 s, 30.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:49:57,469::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:49:57,996::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:49:58,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0386389 s, 26.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:50:00,102::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:50:00,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374102 s, 27.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:50:00,486::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:50:00,686::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 41830dad-f012-4b4d-b636-863f66d5ca4b >Thread-322::DEBUG::2014-05-13 11:50:00,686::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-322::DEBUG::2014-05-13 11:50:00,689::blockSD::336::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:50:00,696::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000627116 s, 3.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:50:00,697::misc::287::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000627116 s, 3.3 MB/s'], size: 2048 >Thread-322::DEBUG::2014-05-13 11:50:00,697::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-322::WARNING::2014-05-13 11:50:00,697::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-322::DEBUG::2014-05-13 11:50:00,697::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-322::DEBUG::2014-05-13 11:50:00,697::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:50:00,698::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:50:00,698::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:50:00,698::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:50:00,699::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:50:00,699::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 11:50:00,927::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:50:00,928::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:50:00,929::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'LOCKPOLICY=', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', 'DESCRIPTION=ISCSI', 'LOGBLKSIZE=512', 'TYPE=ISCSI', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'PHYBLKSIZE=512', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', 'CLASS=Data', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'POOL_DESCRIPTION=DC_2', 'POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'MASTER_VERSION=1', 'ROLE=Master', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', '_SHA_CKSUM=28bb2d7cc60e0d779eb69fe93d050c16c769f069', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0'] >Thread-322::INFO::2014-05-13 11:50:00,930::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_imageNS already registered >Thread-322::INFO::2014-05-13 11:50:00,930::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_volumeNS already registered >Thread-322::INFO::2014-05-13 11:50:00,930::blockSD::452::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_lvmActivationNS already registered >Thread-322::DEBUG::2014-05-13 11:50:00,931::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 11:50:01,097::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:50:01,101::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:50:01,109::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000605195 s, 6.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:50:02,195::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:50:02,275::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373586 s, 27.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:50:03,157::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:50:03,158::task::595::TaskManager.Task::(_updateState) Task=`a080ace1-d2dc-4e8f-a77f-558c18a04367`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:50:03,158::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:50:03,158::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:50:03,159::task::1185::TaskManager.Task::(prepare) Task=`a080ace1-d2dc-4e8f-a77f-558c18a04367`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:50:03,159::task::595::TaskManager.Task::(_updateState) Task=`a080ace1-d2dc-4e8f-a77f-558c18a04367`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:50:03,159::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:50:03,159::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:50:03,159::task::990::TaskManager.Task::(_decref) Task=`a080ace1-d2dc-4e8f-a77f-558c18a04367`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:50:03,167::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:50:03,168::task::595::TaskManager.Task::(_updateState) Task=`90ef66ac-467e-4345-b852-4a7e95eb98a4`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:50:03,168::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:50:03,169::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`07504d05-55af-469f-90e9-ccd2d37b11de`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:50:03,169::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:50:03,170::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:50:03,170::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`07504d05-55af-469f-90e9-ccd2d37b11de`::Granted request >Thread-338::DEBUG::2014-05-13 11:50:03,170::task::827::TaskManager.Task::(resourceAcquired) Task=`90ef66ac-467e-4345-b852-4a7e95eb98a4`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:50:03,170::task::990::TaskManager.Task::(_decref) Task=`90ef66ac-467e-4345-b852-4a7e95eb98a4`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:50:03,171::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:50:03,171::task::1185::TaskManager.Task::(prepare) Task=`90ef66ac-467e-4345-b852-4a7e95eb98a4`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:50:03,171::task::595::TaskManager.Task::(_updateState) Task=`90ef66ac-467e-4345-b852-4a7e95eb98a4`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:50:03,172::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:50:03,172::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:50:03,172::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:50:03,172::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:50:03,172::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:50:03,173::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:50:03,173::task::990::TaskManager.Task::(_decref) Task=`90ef66ac-467e-4345-b852-4a7e95eb98a4`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:50:03,503::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:50:04,288::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:50:04,363::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316718 s, 32.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:50:06,376::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:50:06,453::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321834 s, 31.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:50:06,521::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:50:08,466::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:50:08,844::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.334255 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:50:09,546::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:50:10,857::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 11:50:10,908::task::595::TaskManager.Task::(_updateState) Task=`7360baba-9e09-4bed-88e8-3ab2ccfcaba9`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:50:10,909::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:50:10,910::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000605195', 'lastCheck': '9.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:50:10,911::task::1185::TaskManager.Task::(prepare) Task=`7360baba-9e09-4bed-88e8-3ab2ccfcaba9`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000605195', 'lastCheck': '9.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:50:10,912::task::595::TaskManager.Task::(_updateState) Task=`7360baba-9e09-4bed-88e8-3ab2ccfcaba9`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:50:10,913::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:50:10,914::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:50:10,915::task::990::TaskManager.Task::(_decref) Task=`7360baba-9e09-4bed-88e8-3ab2ccfcaba9`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:50:10,960::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318989 s, 32.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:50:11,116::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:50:11,123::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000660109 s, 6.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:50:12,565::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:50:12,974::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:50:13,057::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.039828 s, 25.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:50:13,193::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:50:13,194::task::595::TaskManager.Task::(_updateState) Task=`2f42cc3a-5e16-4edb-afa6-e113339f9031`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:50:13,194::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:50:13,194::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:50:13,194::task::1185::TaskManager.Task::(prepare) Task=`2f42cc3a-5e16-4edb-afa6-e113339f9031`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:50:13,194::task::595::TaskManager.Task::(_updateState) Task=`2f42cc3a-5e16-4edb-afa6-e113339f9031`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:50:13,195::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:50:13,195::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:50:13,195::task::990::TaskManager.Task::(_decref) Task=`2f42cc3a-5e16-4edb-afa6-e113339f9031`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:50:13,199::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:50:13,199::task::595::TaskManager.Task::(_updateState) Task=`23f1f587-8822-49c1-a502-a2e025696934`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:50:13,200::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:50:13,200::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a69074b5-8867-4ed1-87a8-67e67c4a8b82`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:50:13,200::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:50:13,201::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:50:13,201::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a69074b5-8867-4ed1-87a8-67e67c4a8b82`::Granted request >Thread-338::DEBUG::2014-05-13 11:50:13,201::task::827::TaskManager.Task::(resourceAcquired) Task=`23f1f587-8822-49c1-a502-a2e025696934`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:50:13,201::task::990::TaskManager.Task::(_decref) Task=`23f1f587-8822-49c1-a502-a2e025696934`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:50:13,202::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:50:13,202::task::1185::TaskManager.Task::(prepare) Task=`23f1f587-8822-49c1-a502-a2e025696934`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:50:13,202::task::595::TaskManager.Task::(_updateState) Task=`23f1f587-8822-49c1-a502-a2e025696934`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:50:13,203::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:50:13,203::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:50:13,203::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:50:13,203::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:50:13,203::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:50:13,204::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:50:13,204::task::990::TaskManager.Task::(_decref) Task=`23f1f587-8822-49c1-a502-a2e025696934`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:50:15,070::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:50:15,151::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379853 s, 27.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:50:15,583::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:50:17,164::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:50:17,240::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320522 s, 31.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:50:18,610::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:50:19,253::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:50:19,339::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0435338 s, 23.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:50:21,130::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:50:21,138::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00067399 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:50:21,352::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:50:21,428::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312296 s, 32.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:50:21,628::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:50:23,223::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:50:23,223::task::595::TaskManager.Task::(_updateState) Task=`27e4aaed-c536-4dd0-a744-5e6d40e4e0a9`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:50:23,223::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:50:23,224::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:50:23,224::task::1185::TaskManager.Task::(prepare) Task=`27e4aaed-c536-4dd0-a744-5e6d40e4e0a9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:50:23,224::task::595::TaskManager.Task::(_updateState) Task=`27e4aaed-c536-4dd0-a744-5e6d40e4e0a9`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:50:23,224::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:50:23,224::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:50:23,225::task::990::TaskManager.Task::(_decref) Task=`27e4aaed-c536-4dd0-a744-5e6d40e4e0a9`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:50:23,229::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:50:23,230::task::595::TaskManager.Task::(_updateState) Task=`7e844946-021e-4285-a1d7-1d6c25ac360b`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:50:23,230::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:50:23,230::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`05ce9f41-b48a-44ac-988b-6514758035e2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:50:23,231::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:50:23,231::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:50:23,231::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`05ce9f41-b48a-44ac-988b-6514758035e2`::Granted request >Thread-338::DEBUG::2014-05-13 11:50:23,231::task::827::TaskManager.Task::(resourceAcquired) Task=`7e844946-021e-4285-a1d7-1d6c25ac360b`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:50:23,232::task::990::TaskManager.Task::(_decref) Task=`7e844946-021e-4285-a1d7-1d6c25ac360b`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:50:23,232::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:50:23,232::task::1185::TaskManager.Task::(prepare) Task=`7e844946-021e-4285-a1d7-1d6c25ac360b`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:50:23,233::task::595::TaskManager.Task::(_updateState) Task=`7e844946-021e-4285-a1d7-1d6c25ac360b`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:50:23,233::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:50:23,233::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:50:23,233::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:50:23,233::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:50:23,234::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:50:23,234::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:50:23,234::task::990::TaskManager.Task::(_decref) Task=`7e844946-021e-4285-a1d7-1d6c25ac360b`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:50:23,441::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:50:23,523::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.039557 s, 25.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:50:24,647::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:50:25,536::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:50:25,610::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307802 s, 33.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:50:26,064::task::595::TaskManager.Task::(_updateState) Task=`eda82ed0-66cc-4018-a445-c9a47822c8e5`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:50:26,064::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:50:26,065::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00067399', 'lastCheck': '4.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:50:26,065::task::1185::TaskManager.Task::(prepare) Task=`eda82ed0-66cc-4018-a445-c9a47822c8e5`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00067399', 'lastCheck': '4.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:50:26,065::task::595::TaskManager.Task::(_updateState) Task=`eda82ed0-66cc-4018-a445-c9a47822c8e5`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:50:26,065::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:50:26,065::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:50:26,066::task::990::TaskManager.Task::(_decref) Task=`eda82ed0-66cc-4018-a445-c9a47822c8e5`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:50:27,623::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:50:27,665::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:50:27,702::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321737 s, 31.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:50:29,716::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:50:29,799::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383388 s, 26.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:50:30,684::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:50:31,145::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:50:31,153::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000649938 s, 6.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:50:31,812::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:50:31,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0385606 s, 26.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:50:33,253::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17df19a0] >Thread-13::DEBUG::2014-05-13 11:50:33,253::task::595::TaskManager.Task::(_updateState) Task=`aeb42ba4-11da-4aee-b54e-74e84efc24df`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:50:33,253::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:50:33,254::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:50:33,254::task::1185::TaskManager.Task::(prepare) Task=`aeb42ba4-11da-4aee-b54e-74e84efc24df`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:50:33,254::task::595::TaskManager.Task::(_updateState) Task=`aeb42ba4-11da-4aee-b54e-74e84efc24df`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:50:33,254::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:50:33,254::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:50:33,255::task::990::TaskManager.Task::(_decref) Task=`aeb42ba4-11da-4aee-b54e-74e84efc24df`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:50:33,260::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17df19a0] >Thread-338::DEBUG::2014-05-13 11:50:33,260::task::595::TaskManager.Task::(_updateState) Task=`72814519-d59c-49a3-b5aa-2c9fab7320d9`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:50:33,260::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:50:33,261::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3de5ba70-926d-4361-af3d-ddccebf3144c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:50:33,261::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:50:33,261::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:50:33,261::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3de5ba70-926d-4361-af3d-ddccebf3144c`::Granted request >Thread-338::DEBUG::2014-05-13 11:50:33,262::task::827::TaskManager.Task::(resourceAcquired) Task=`72814519-d59c-49a3-b5aa-2c9fab7320d9`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:50:33,262::task::990::TaskManager.Task::(_decref) Task=`72814519-d59c-49a3-b5aa-2c9fab7320d9`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:50:33,263::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:50:33,263::task::1185::TaskManager.Task::(prepare) Task=`72814519-d59c-49a3-b5aa-2c9fab7320d9`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:50:33,263::task::595::TaskManager.Task::(_updateState) Task=`72814519-d59c-49a3-b5aa-2c9fab7320d9`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:50:33,263::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:50:33,263::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:50:33,264::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:50:33,264::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:50:33,264::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:50:33,264::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:50:33,264::task::990::TaskManager.Task::(_decref) Task=`72814519-d59c-49a3-b5aa-2c9fab7320d9`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:50:33,706::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:50:33,907::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:50:33,982::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318118 s, 32.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:50:35,995::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:50:36,369::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.330175 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:50:36,728::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:50:38,377::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:50:38,456::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355418 s, 28.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:50:39,746::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:50:40,469::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:50:40,544::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0336562 s, 30.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:50:41,160::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:50:41,168::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00074461 s, 5.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:50:41,232::task::595::TaskManager.Task::(_updateState) Task=`50a0bab3-41d6-44df-8ab4-6a3bf0a95d3b`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:50:41,233::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:50:41,233::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00074461', 'lastCheck': '0.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:50:41,233::task::1185::TaskManager.Task::(prepare) Task=`50a0bab3-41d6-44df-8ab4-6a3bf0a95d3b`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00074461', 'lastCheck': '0.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:50:41,233::task::595::TaskManager.Task::(_updateState) Task=`50a0bab3-41d6-44df-8ab4-6a3bf0a95d3b`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:50:41,233::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:50:41,234::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:50:41,234::task::990::TaskManager.Task::(_decref) Task=`50a0bab3-41d6-44df-8ab4-6a3bf0a95d3b`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:50:42,557::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:50:42,633::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0334981 s, 30.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:50:42,767::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:50:43,292::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [50b954f9] >Thread-13::DEBUG::2014-05-13 11:50:43,293::task::595::TaskManager.Task::(_updateState) Task=`845e7957-6666-491a-8971-a617092e2cfa`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:50:43,293::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:50:43,293::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:50:43,294::task::1185::TaskManager.Task::(prepare) Task=`845e7957-6666-491a-8971-a617092e2cfa`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:50:43,294::task::595::TaskManager.Task::(_updateState) Task=`845e7957-6666-491a-8971-a617092e2cfa`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:50:43,294::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:50:43,294::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:50:43,294::task::990::TaskManager.Task::(_decref) Task=`845e7957-6666-491a-8971-a617092e2cfa`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:50:43,299::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [50b954f9] >Thread-338::DEBUG::2014-05-13 11:50:43,299::task::595::TaskManager.Task::(_updateState) Task=`38a696a1-3268-4598-9740-ab978367296f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:50:43,299::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:50:43,300::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9b3bcc85-21eb-468b-a51a-edb0428017b8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:50:43,300::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:50:43,300::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:50:43,300::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9b3bcc85-21eb-468b-a51a-edb0428017b8`::Granted request >Thread-338::DEBUG::2014-05-13 11:50:43,301::task::827::TaskManager.Task::(resourceAcquired) Task=`38a696a1-3268-4598-9740-ab978367296f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:50:43,301::task::990::TaskManager.Task::(_decref) Task=`38a696a1-3268-4598-9740-ab978367296f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:50:43,302::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:50:43,302::task::1185::TaskManager.Task::(prepare) Task=`38a696a1-3268-4598-9740-ab978367296f`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:50:43,302::task::595::TaskManager.Task::(_updateState) Task=`38a696a1-3268-4598-9740-ab978367296f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:50:43,302::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:50:43,302::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:50:43,303::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:50:43,303::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:50:43,303::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:50:43,303::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:50:43,303::task::990::TaskManager.Task::(_decref) Task=`38a696a1-3268-4598-9740-ab978367296f`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:50:44,646::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:50:45,044::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.355468 s, 2.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:50:45,796::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:50:47,057::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:50:47,133::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0336659 s, 30.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:50:48,814::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:50:49,146::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:50:49,226::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0344581 s, 29.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:50:51,197::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:50:51,205::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000765123 s, 5.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:50:51,240::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:50:51,320::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365852 s, 28.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:50:51,833::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:50:53,321::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:50:53,321::task::595::TaskManager.Task::(_updateState) Task=`5d50db59-f7de-4099-bfe6-117432dce64f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:50:53,322::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:50:53,322::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:50:53,322::task::1185::TaskManager.Task::(prepare) Task=`5d50db59-f7de-4099-bfe6-117432dce64f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:50:53,322::task::595::TaskManager.Task::(_updateState) Task=`5d50db59-f7de-4099-bfe6-117432dce64f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:50:53,323::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:50:53,323::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:50:53,323::task::990::TaskManager.Task::(_decref) Task=`5d50db59-f7de-4099-bfe6-117432dce64f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:50:53,328::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:50:53,328::task::595::TaskManager.Task::(_updateState) Task=`8b23db27-51c0-4b96-8681-00d35246ebff`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:50:53,328::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:50:53,329::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ce4081c6-63c1-4403-a631-e7f5aca59c2e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:50:53,329::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:50:53,329::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:50:53,329::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ce4081c6-63c1-4403-a631-e7f5aca59c2e`::Granted request >Thread-338::DEBUG::2014-05-13 11:50:53,330::task::827::TaskManager.Task::(resourceAcquired) Task=`8b23db27-51c0-4b96-8681-00d35246ebff`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:50:53,330::task::990::TaskManager.Task::(_decref) Task=`8b23db27-51c0-4b96-8681-00d35246ebff`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:50:53,331::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:50:53,331::task::1185::TaskManager.Task::(prepare) Task=`8b23db27-51c0-4b96-8681-00d35246ebff`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:50:53,331::task::595::TaskManager.Task::(_updateState) Task=`8b23db27-51c0-4b96-8681-00d35246ebff`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:50:53,331::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:50:53,331::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:50:53,332::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:50:53,332::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:50:53,332::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:50:53,332::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:50:53,333::task::990::TaskManager.Task::(_decref) Task=`8b23db27-51c0-4b96-8681-00d35246ebff`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:50:53,334::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:50:53,419::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0403463 s, 25.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:50:54,852::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:50:55,432::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:50:55,837::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.362593 s, 2.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:50:56,409::task::595::TaskManager.Task::(_updateState) Task=`be399256-bbef-407e-9a9f-2fbd541193a4`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:50:56,410::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:50:56,410::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000765123', 'lastCheck': '5.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:50:56,410::task::1185::TaskManager.Task::(prepare) Task=`be399256-bbef-407e-9a9f-2fbd541193a4`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000765123', 'lastCheck': '5.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:50:56,410::task::595::TaskManager.Task::(_updateState) Task=`be399256-bbef-407e-9a9f-2fbd541193a4`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:50:56,410::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:50:56,410::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:50:56,411::task::990::TaskManager.Task::(_decref) Task=`be399256-bbef-407e-9a9f-2fbd541193a4`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:50:57,850::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:50:57,877::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:50:57,926::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0332488 s, 30.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:50:59,939::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:00,018::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358179 s, 28.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:51:00,895::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:51:01,212::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:51:01,220::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000589787 s, 6.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:51:02,031::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:02,110::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357484 s, 28.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:51:03,368::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [7a000b17] >Thread-13::DEBUG::2014-05-13 11:51:03,369::task::595::TaskManager.Task::(_updateState) Task=`99a74ec1-0ac3-4a0a-ac17-d6963492b8be`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:51:03,369::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:51:03,369::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:51:03,369::task::1185::TaskManager.Task::(prepare) Task=`99a74ec1-0ac3-4a0a-ac17-d6963492b8be`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:51:03,370::task::595::TaskManager.Task::(_updateState) Task=`99a74ec1-0ac3-4a0a-ac17-d6963492b8be`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:51:03,370::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:51:03,370::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:51:03,370::task::990::TaskManager.Task::(_decref) Task=`99a74ec1-0ac3-4a0a-ac17-d6963492b8be`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:51:03,375::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [7a000b17] >Thread-338::DEBUG::2014-05-13 11:51:03,375::task::595::TaskManager.Task::(_updateState) Task=`bffdcea7-4d8b-44e9-a583-e2e04c5d594b`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:51:03,375::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:51:03,376::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c4acb1bb-cb31-4f32-acb5-34b34f8f4cb2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:51:03,376::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:51:03,376::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:51:03,376::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c4acb1bb-cb31-4f32-acb5-34b34f8f4cb2`::Granted request >Thread-338::DEBUG::2014-05-13 11:51:03,376::task::827::TaskManager.Task::(resourceAcquired) Task=`bffdcea7-4d8b-44e9-a583-e2e04c5d594b`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:51:03,377::task::990::TaskManager.Task::(_decref) Task=`bffdcea7-4d8b-44e9-a583-e2e04c5d594b`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:51:03,377::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:51:03,377::task::1185::TaskManager.Task::(prepare) Task=`bffdcea7-4d8b-44e9-a583-e2e04c5d594b`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:51:03,377::task::595::TaskManager.Task::(_updateState) Task=`bffdcea7-4d8b-44e9-a583-e2e04c5d594b`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:51:03,378::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:51:03,378::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:51:03,378::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:51:03,378::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:51:03,378::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:51:03,378::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:51:03,379::task::990::TaskManager.Task::(_decref) Task=`bffdcea7-4d8b-44e9-a583-e2e04c5d594b`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:51:03,915::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:51:04,123::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:04,199::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319601 s, 32.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:51:06,212::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:06,308::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0514889 s, 19.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:51:06,937::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:51:08,321::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:08,401::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384229 s, 26.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:51:09,958::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:51:10,414::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:10,504::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0469811 s, 21.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:51:11,228::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:51:11,236::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00069072 s, 5.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:51:11,553::task::595::TaskManager.Task::(_updateState) Task=`30dedd1a-3331-4e7e-96eb-f11756b17786`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:51:11,553::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:51:11,553::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00069072', 'lastCheck': '0.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:51:11,554::task::1185::TaskManager.Task::(prepare) Task=`30dedd1a-3331-4e7e-96eb-f11756b17786`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00069072', 'lastCheck': '0.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:51:11,554::task::595::TaskManager.Task::(_updateState) Task=`30dedd1a-3331-4e7e-96eb-f11756b17786`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:51:11,554::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:51:11,554::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:51:11,554::task::990::TaskManager.Task::(_decref) Task=`30dedd1a-3331-4e7e-96eb-f11756b17786`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:51:12,517::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:12,593::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0327249 s, 31.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:51:12,983::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:51:13,398::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:51:13,398::task::595::TaskManager.Task::(_updateState) Task=`d14836e6-6e23-44aa-829c-9e95246d85fa`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:51:13,398::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:51:13,399::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:51:13,399::task::1185::TaskManager.Task::(prepare) Task=`d14836e6-6e23-44aa-829c-9e95246d85fa`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:51:13,399::task::595::TaskManager.Task::(_updateState) Task=`d14836e6-6e23-44aa-829c-9e95246d85fa`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:51:13,399::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:51:13,400::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:51:13,400::task::990::TaskManager.Task::(_decref) Task=`d14836e6-6e23-44aa-829c-9e95246d85fa`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:51:13,404::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:51:13,404::task::595::TaskManager.Task::(_updateState) Task=`b31b20d4-a118-411f-b652-06a9bd63f503`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:51:13,405::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:51:13,405::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`308492ca-2fd4-4da8-8300-956f132bc750`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:51:13,405::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:51:13,405::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:51:13,406::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`308492ca-2fd4-4da8-8300-956f132bc750`::Granted request >Thread-338::DEBUG::2014-05-13 11:51:13,406::task::827::TaskManager.Task::(resourceAcquired) Task=`b31b20d4-a118-411f-b652-06a9bd63f503`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:51:13,406::task::990::TaskManager.Task::(_decref) Task=`b31b20d4-a118-411f-b652-06a9bd63f503`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:51:13,407::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:51:13,407::task::1185::TaskManager.Task::(prepare) Task=`b31b20d4-a118-411f-b652-06a9bd63f503`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:51:13,407::task::595::TaskManager.Task::(_updateState) Task=`b31b20d4-a118-411f-b652-06a9bd63f503`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:51:13,408::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:51:13,408::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:51:13,408::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:51:13,408::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:51:13,408::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:51:13,408::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:51:13,409::task::990::TaskManager.Task::(_decref) Task=`b31b20d4-a118-411f-b652-06a9bd63f503`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:51:14,606::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:14,692::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0440558 s, 23.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:51:16,002::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:51:16,705::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:16,779::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322472 s, 31.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:51:18,792::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:18,868::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0337245 s, 30.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:51:19,021::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:51:20,881::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:20,961::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363585 s, 28.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:51:21,285::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:51:21,293::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000689906 s, 5.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:51:22,041::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:51:22,973::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:23,415::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.398439 s, 2.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:51:23,427::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [19876d15] >Thread-13::DEBUG::2014-05-13 11:51:23,428::task::595::TaskManager.Task::(_updateState) Task=`b6c525bb-dee1-487d-808f-6462b2adff34`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:51:23,429::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:51:23,430::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:51:23,431::task::1185::TaskManager.Task::(prepare) Task=`b6c525bb-dee1-487d-808f-6462b2adff34`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:51:23,432::task::595::TaskManager.Task::(_updateState) Task=`b6c525bb-dee1-487d-808f-6462b2adff34`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:51:23,432::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:51:23,432::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:51:23,432::task::990::TaskManager.Task::(_decref) Task=`b6c525bb-dee1-487d-808f-6462b2adff34`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:51:23,437::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [19876d15] >Thread-338::DEBUG::2014-05-13 11:51:23,437::task::595::TaskManager.Task::(_updateState) Task=`e64d95d3-0186-4e5b-a1ac-b0256e033256`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:51:23,438::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:51:23,438::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`235f4860-7097-4507-8266-f7557609c288`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:51:23,438::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:51:23,439::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:51:23,439::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`235f4860-7097-4507-8266-f7557609c288`::Granted request >Thread-338::DEBUG::2014-05-13 11:51:23,439::task::827::TaskManager.Task::(resourceAcquired) Task=`e64d95d3-0186-4e5b-a1ac-b0256e033256`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:51:23,439::task::990::TaskManager.Task::(_decref) Task=`e64d95d3-0186-4e5b-a1ac-b0256e033256`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:51:23,440::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:51:23,440::task::1185::TaskManager.Task::(prepare) Task=`e64d95d3-0186-4e5b-a1ac-b0256e033256`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:51:23,440::task::595::TaskManager.Task::(_updateState) Task=`e64d95d3-0186-4e5b-a1ac-b0256e033256`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:51:23,441::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:51:23,441::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:51:23,441::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:51:23,441::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:51:23,441::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:51:23,441::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:51:23,442::task::990::TaskManager.Task::(_decref) Task=`e64d95d3-0186-4e5b-a1ac-b0256e033256`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:51:25,060::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:51:25,434::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:25,509::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317152 s, 32.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:51:26,694::task::595::TaskManager.Task::(_updateState) Task=`6966d34e-b4cb-4eef-8482-1810bee5632d`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:51:26,694::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:51:26,694::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000689906', 'lastCheck': '5.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:51:26,694::task::1185::TaskManager.Task::(prepare) Task=`6966d34e-b4cb-4eef-8482-1810bee5632d`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000689906', 'lastCheck': '5.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:51:26,694::task::595::TaskManager.Task::(_updateState) Task=`6966d34e-b4cb-4eef-8482-1810bee5632d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:51:26,695::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:51:26,695::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:51:26,695::task::990::TaskManager.Task::(_decref) Task=`6966d34e-b4cb-4eef-8482-1810bee5632d`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:51:27,522::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:27,597::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323143 s, 31.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:51:28,080::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:51:29,610::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:29,692::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387628 s, 26.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:51:31,097::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:51:31,300::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:51:31,308::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000645979 s, 6.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:51:31,705::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:31,788::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0395585 s, 25.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:51:33,460::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:51:33,461::task::595::TaskManager.Task::(_updateState) Task=`9c18e3c8-46c7-49e9-b9c0-71751bec2d76`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:51:33,461::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:51:33,461::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:51:33,461::task::1185::TaskManager.Task::(prepare) Task=`9c18e3c8-46c7-49e9-b9c0-71751bec2d76`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:51:33,462::task::595::TaskManager.Task::(_updateState) Task=`9c18e3c8-46c7-49e9-b9c0-71751bec2d76`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:51:33,462::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:51:33,462::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:51:33,462::task::990::TaskManager.Task::(_decref) Task=`9c18e3c8-46c7-49e9-b9c0-71751bec2d76`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:51:33,467::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:51:33,467::task::595::TaskManager.Task::(_updateState) Task=`6fd96c28-550c-4922-a2dd-47da334382b3`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:51:33,467::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:51:33,468::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`75c73ef9-0eb2-4898-a97f-f963c0a7248d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:51:33,468::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:51:33,468::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:51:33,468::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`75c73ef9-0eb2-4898-a97f-f963c0a7248d`::Granted request >Thread-338::DEBUG::2014-05-13 11:51:33,469::task::827::TaskManager.Task::(resourceAcquired) Task=`6fd96c28-550c-4922-a2dd-47da334382b3`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:51:33,469::task::990::TaskManager.Task::(_decref) Task=`6fd96c28-550c-4922-a2dd-47da334382b3`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:51:33,470::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:51:33,470::task::1185::TaskManager.Task::(prepare) Task=`6fd96c28-550c-4922-a2dd-47da334382b3`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:51:33,470::task::595::TaskManager.Task::(_updateState) Task=`6fd96c28-550c-4922-a2dd-47da334382b3`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:51:33,470::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:51:33,470::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:51:33,470::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:51:33,471::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:51:33,471::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:51:33,471::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:51:33,471::task::990::TaskManager.Task::(_decref) Task=`6fd96c28-550c-4922-a2dd-47da334382b3`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:51:33,801::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:33,877::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0342572 s, 29.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:51:34,115::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:51:35,890::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:35,985::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.052992 s, 19.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:51:37,134::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:51:37,998::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:38,073::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309523 s, 33.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:51:40,086::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:51:40,153::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:51:40,165::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318557 s, 32.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:51:41,315::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:51:41,324::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00152758 s, 2.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:51:41,836::task::595::TaskManager.Task::(_updateState) Task=`f91338cd-2938-4291-82aa-81a2186f2b25`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:51:41,836::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:51:41,836::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00152758', 'lastCheck': '0.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:51:41,836::task::1185::TaskManager.Task::(prepare) Task=`f91338cd-2938-4291-82aa-81a2186f2b25`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00152758', 'lastCheck': '0.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:51:41,837::task::595::TaskManager.Task::(_updateState) Task=`f91338cd-2938-4291-82aa-81a2186f2b25`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:51:41,837::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:51:41,837::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:51:41,837::task::990::TaskManager.Task::(_decref) Task=`f91338cd-2938-4291-82aa-81a2186f2b25`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:51:42,178::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:42,258::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381583 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:51:43,171::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:51:43,499::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5c06d0d6] >Thread-13::DEBUG::2014-05-13 11:51:43,500::task::595::TaskManager.Task::(_updateState) Task=`34fdc263-0400-468e-9988-2b2c00b1f298`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:51:43,500::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:51:43,501::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:51:43,501::task::1185::TaskManager.Task::(prepare) Task=`34fdc263-0400-468e-9988-2b2c00b1f298`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:51:43,501::task::595::TaskManager.Task::(_updateState) Task=`34fdc263-0400-468e-9988-2b2c00b1f298`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:51:43,501::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:51:43,502::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:51:43,502::task::990::TaskManager.Task::(_decref) Task=`34fdc263-0400-468e-9988-2b2c00b1f298`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:51:43,513::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5c06d0d6] >Thread-338::DEBUG::2014-05-13 11:51:43,514::task::595::TaskManager.Task::(_updateState) Task=`85310d54-b44d-4daf-8f00-aa62c761ebed`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:51:43,515::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:51:43,515::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`053be6e2-c662-422d-ae52-158067493da2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:51:43,516::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:51:43,516::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:51:43,516::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`053be6e2-c662-422d-ae52-158067493da2`::Granted request >Thread-338::DEBUG::2014-05-13 11:51:43,516::task::827::TaskManager.Task::(resourceAcquired) Task=`85310d54-b44d-4daf-8f00-aa62c761ebed`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:51:43,517::task::990::TaskManager.Task::(_decref) Task=`85310d54-b44d-4daf-8f00-aa62c761ebed`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:51:43,518::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:51:43,518::task::1185::TaskManager.Task::(prepare) Task=`85310d54-b44d-4daf-8f00-aa62c761ebed`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:51:43,518::task::595::TaskManager.Task::(_updateState) Task=`85310d54-b44d-4daf-8f00-aa62c761ebed`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:51:43,518::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:51:43,518::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:51:43,519::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:51:43,519::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:51:43,519::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:51:43,519::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:51:43,520::task::990::TaskManager.Task::(_decref) Task=`85310d54-b44d-4daf-8f00-aa62c761ebed`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:51:44,271::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:44,352::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381035 s, 26.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:51:46,189::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:51:46,376::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:46,454::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0326254 s, 31.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:51:48,468::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:48,546::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322863 s, 31.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:51:49,208::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:51:50,559::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:50,976::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.369922 s, 2.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:51:51,337::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:51:51,345::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000780945 s, 5.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:51:52,227::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:51:52,989::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:53,064::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320654 s, 31.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:51:53,538::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4dbe430a] >Thread-13::DEBUG::2014-05-13 11:51:53,538::task::595::TaskManager.Task::(_updateState) Task=`3aab07e2-bb46-43d5-8f3a-d5e93b130ecc`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:51:53,539::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:51:53,539::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:51:53,539::task::1185::TaskManager.Task::(prepare) Task=`3aab07e2-bb46-43d5-8f3a-d5e93b130ecc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:51:53,539::task::595::TaskManager.Task::(_updateState) Task=`3aab07e2-bb46-43d5-8f3a-d5e93b130ecc`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:51:53,539::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:51:53,540::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:51:53,540::task::990::TaskManager.Task::(_decref) Task=`3aab07e2-bb46-43d5-8f3a-d5e93b130ecc`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:51:53,544::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4dbe430a] >Thread-338::DEBUG::2014-05-13 11:51:53,545::task::595::TaskManager.Task::(_updateState) Task=`5002e0de-006c-4213-b576-ea0c78bd0879`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:51:53,545::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:51:53,545::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ca3b5544-eb65-4e11-ba08-8503d38821b3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:51:53,545::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:51:53,546::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:51:53,546::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ca3b5544-eb65-4e11-ba08-8503d38821b3`::Granted request >Thread-338::DEBUG::2014-05-13 11:51:53,546::task::827::TaskManager.Task::(resourceAcquired) Task=`5002e0de-006c-4213-b576-ea0c78bd0879`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:51:53,546::task::990::TaskManager.Task::(_decref) Task=`5002e0de-006c-4213-b576-ea0c78bd0879`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:51:53,547::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:51:53,547::task::1185::TaskManager.Task::(prepare) Task=`5002e0de-006c-4213-b576-ea0c78bd0879`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:51:53,548::task::595::TaskManager.Task::(_updateState) Task=`5002e0de-006c-4213-b576-ea0c78bd0879`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:51:53,548::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:51:53,548::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:51:53,548::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:51:53,548::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:51:53,548::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:51:53,549::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:51:53,549::task::990::TaskManager.Task::(_decref) Task=`5002e0de-006c-4213-b576-ea0c78bd0879`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:51:55,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:55,152::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318923 s, 32.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:51:55,246::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:51:56,976::task::595::TaskManager.Task::(_updateState) Task=`ab098672-ca52-42c4-bd38-b7a9cb7058b6`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:51:56,977::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:51:56,977::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000780945', 'lastCheck': '5.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:51:56,977::task::1185::TaskManager.Task::(prepare) Task=`ab098672-ca52-42c4-bd38-b7a9cb7058b6`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000780945', 'lastCheck': '5.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:51:56,977::task::595::TaskManager.Task::(_updateState) Task=`ab098672-ca52-42c4-bd38-b7a9cb7058b6`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:51:56,977::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:51:56,977::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:51:56,978::task::990::TaskManager.Task::(_decref) Task=`ab098672-ca52-42c4-bd38-b7a9cb7058b6`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:51:57,165::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:57,244::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361302 s, 28.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:51:58,267::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:51:59,258::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:51:59,340::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0395753 s, 25.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:52:01,285::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:52:01,353::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 11:52:01,353::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:52:01,389::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.0148239 s, 276 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:52:01,474::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0790017 s, 13.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:52:03,487::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 11:52:03,570::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Dummy-337::DEBUG::2014-05-13 11:52:03,571::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383366 s, 26.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:52:03,573::task::595::TaskManager.Task::(_updateState) Task=`01bc30d2-4a5c-46f0-9b18-0c4066e87c1a`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:52:03,574::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:52:03,575::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:52:03,576::task::1185::TaskManager.Task::(prepare) Task=`01bc30d2-4a5c-46f0-9b18-0c4066e87c1a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:52:03,577::task::595::TaskManager.Task::(_updateState) Task=`01bc30d2-4a5c-46f0-9b18-0c4066e87c1a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:52:03,578::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:52:03,578::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:52:03,579::task::990::TaskManager.Task::(_decref) Task=`01bc30d2-4a5c-46f0-9b18-0c4066e87c1a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:52:03,587::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:52:03,588::task::595::TaskManager.Task::(_updateState) Task=`efe91107-415e-4c6b-a80d-6e0ff7fe3b1d`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:52:03,589::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:52:03,591::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5bbe54f1-0bb8-4f6e-bd10-73969420b89e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:52:03,592::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:52:03,593::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:52:03,594::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5bbe54f1-0bb8-4f6e-bd10-73969420b89e`::Granted request >Thread-338::DEBUG::2014-05-13 11:52:03,595::task::827::TaskManager.Task::(resourceAcquired) Task=`efe91107-415e-4c6b-a80d-6e0ff7fe3b1d`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:52:03,596::task::990::TaskManager.Task::(_decref) Task=`efe91107-415e-4c6b-a80d-6e0ff7fe3b1d`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:52:03,599::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:52:03,600::task::1185::TaskManager.Task::(prepare) Task=`efe91107-415e-4c6b-a80d-6e0ff7fe3b1d`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:52:03,601::task::595::TaskManager.Task::(_updateState) Task=`efe91107-415e-4c6b-a80d-6e0ff7fe3b1d`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:52:03,601::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:52:03,602::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:52:03,603::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:52:03,604::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:52:03,605::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:52:03,606::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:52:03,607::task::990::TaskManager.Task::(_decref) Task=`efe91107-415e-4c6b-a80d-6e0ff7fe3b1d`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:52:04,303::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:52:05,617::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:52:05,695::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360909 s, 28.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:52:07,321::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:52:07,709::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:52:07,786::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0336 s, 30.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:52:09,799::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:52:09,873::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0324519 s, 31.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:52:10,340::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:52:11,417::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:52:11,425::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000730464 s, 5.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:52:11,886::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:52:11,966::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382737 s, 26.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:52:12,116::task::595::TaskManager.Task::(_updateState) Task=`a9e16ba2-c9d9-4e34-a07a-3135f83e85e3`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:52:12,116::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:52:12,116::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000730464', 'lastCheck': '0.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:52:12,116::task::1185::TaskManager.Task::(prepare) Task=`a9e16ba2-c9d9-4e34-a07a-3135f83e85e3`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000730464', 'lastCheck': '0.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:52:12,116::task::595::TaskManager.Task::(_updateState) Task=`a9e16ba2-c9d9-4e34-a07a-3135f83e85e3`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:52:12,117::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:52:12,117::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:52:12,117::task::990::TaskManager.Task::(_decref) Task=`a9e16ba2-c9d9-4e34-a07a-3135f83e85e3`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:52:13,370::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:52:13,628::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [12ac301a] >Thread-13::DEBUG::2014-05-13 11:52:13,629::task::595::TaskManager.Task::(_updateState) Task=`7e924ae0-1236-4361-a8ec-e941706d6f1a`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:52:13,629::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:52:13,630::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:52:13,630::task::1185::TaskManager.Task::(prepare) Task=`7e924ae0-1236-4361-a8ec-e941706d6f1a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:52:13,630::task::595::TaskManager.Task::(_updateState) Task=`7e924ae0-1236-4361-a8ec-e941706d6f1a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:52:13,630::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:52:13,630::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:52:13,630::task::990::TaskManager.Task::(_decref) Task=`7e924ae0-1236-4361-a8ec-e941706d6f1a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:52:13,636::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [12ac301a] >Thread-338::DEBUG::2014-05-13 11:52:13,636::task::595::TaskManager.Task::(_updateState) Task=`5bb85fac-a9a3-4240-b1fb-37dc0dcb9097`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:52:13,637::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:52:13,637::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`55b08550-9321-4e59-8167-d414839b8a20`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:52:13,637::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:52:13,637::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:52:13,638::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`55b08550-9321-4e59-8167-d414839b8a20`::Granted request >Thread-338::DEBUG::2014-05-13 11:52:13,638::task::827::TaskManager.Task::(resourceAcquired) Task=`5bb85fac-a9a3-4240-b1fb-37dc0dcb9097`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:52:13,638::task::990::TaskManager.Task::(_decref) Task=`5bb85fac-a9a3-4240-b1fb-37dc0dcb9097`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:52:13,639::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:52:13,639::task::1185::TaskManager.Task::(prepare) Task=`5bb85fac-a9a3-4240-b1fb-37dc0dcb9097`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:52:13,639::task::595::TaskManager.Task::(_updateState) Task=`5bb85fac-a9a3-4240-b1fb-37dc0dcb9097`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:52:13,639::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:52:13,640::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:52:13,640::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:52:13,640::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:52:13,640::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:52:13,640::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:52:13,641::task::990::TaskManager.Task::(_decref) Task=`5bb85fac-a9a3-4240-b1fb-37dc0dcb9097`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:52:13,979::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:52:14,059::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381823 s, 26.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:52:16,072::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:52:16,159::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0455913 s, 22.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:52:16,388::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:52:18,172::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:52:18,258::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0418338 s, 24.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:52:19,408::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:52:20,270::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:52:20,350::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365245 s, 28.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:52:21,432::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:52:21,440::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000676272 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:52:22,363::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:52:22,427::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:52:22,447::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382638 s, 26.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:52:23,659::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5fad7764] >Thread-13::DEBUG::2014-05-13 11:52:23,660::task::595::TaskManager.Task::(_updateState) Task=`89d4de2f-9838-437c-9a27-652d518c5af9`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:52:23,660::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:52:23,660::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:52:23,660::task::1185::TaskManager.Task::(prepare) Task=`89d4de2f-9838-437c-9a27-652d518c5af9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:52:23,661::task::595::TaskManager.Task::(_updateState) Task=`89d4de2f-9838-437c-9a27-652d518c5af9`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:52:23,661::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:52:23,661::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:52:23,661::task::990::TaskManager.Task::(_decref) Task=`89d4de2f-9838-437c-9a27-652d518c5af9`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:52:23,665::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5fad7764] >Thread-338::DEBUG::2014-05-13 11:52:23,666::task::595::TaskManager.Task::(_updateState) Task=`2f029e60-6ea0-428a-ad64-ef670e25a509`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:52:23,666::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:52:23,667::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`77ef1080-d1a6-4a69-9d57-bb38830ed699`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:52:23,668::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:52:23,668::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:52:23,668::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`77ef1080-d1a6-4a69-9d57-bb38830ed699`::Granted request >Thread-338::DEBUG::2014-05-13 11:52:23,668::task::827::TaskManager.Task::(resourceAcquired) Task=`2f029e60-6ea0-428a-ad64-ef670e25a509`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:52:23,669::task::990::TaskManager.Task::(_decref) Task=`2f029e60-6ea0-428a-ad64-ef670e25a509`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:52:23,669::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:52:23,669::task::1185::TaskManager.Task::(prepare) Task=`2f029e60-6ea0-428a-ad64-ef670e25a509`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:52:23,670::task::595::TaskManager.Task::(_updateState) Task=`2f029e60-6ea0-428a-ad64-ef670e25a509`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:52:23,670::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:52:23,670::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:52:23,670::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:52:23,670::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:52:23,671::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:52:23,671::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:52:23,671::task::990::TaskManager.Task::(_decref) Task=`2f029e60-6ea0-428a-ad64-ef670e25a509`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:52:24,460::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:52:24,537::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0331356 s, 30.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:52:25,444::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:52:26,551::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:52:26,637::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0432768 s, 23.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:52:27,251::task::595::TaskManager.Task::(_updateState) Task=`43303dad-3af3-49e3-96b5-600837ec8773`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:52:27,251::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:52:27,251::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000676272', 'lastCheck': '5.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:52:27,252::task::1185::TaskManager.Task::(prepare) Task=`43303dad-3af3-49e3-96b5-600837ec8773`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000676272', 'lastCheck': '5.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:52:27,252::task::595::TaskManager.Task::(_updateState) Task=`43303dad-3af3-49e3-96b5-600837ec8773`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:52:27,252::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:52:27,252::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:52:27,252::task::990::TaskManager.Task::(_decref) Task=`43303dad-3af3-49e3-96b5-600837ec8773`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:52:28,464::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:52:28,650::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:52:28,729::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356708 s, 28.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:52:30,742::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:52:31,113::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.329211 s, 3.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:52:31,447::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:52:31,455::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000667026 s, 6.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:52:31,483::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:52:33,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:52:33,205::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357663 s, 28.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:52:33,689::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:52:33,690::task::595::TaskManager.Task::(_updateState) Task=`3407d223-b559-4b10-951f-155a2a0657de`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:52:33,690::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:52:33,691::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:52:33,691::task::1185::TaskManager.Task::(prepare) Task=`3407d223-b559-4b10-951f-155a2a0657de`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:52:33,691::task::595::TaskManager.Task::(_updateState) Task=`3407d223-b559-4b10-951f-155a2a0657de`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:52:33,691::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:52:33,691::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:52:33,692::task::990::TaskManager.Task::(_decref) Task=`3407d223-b559-4b10-951f-155a2a0657de`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:52:33,696::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:52:33,697::task::595::TaskManager.Task::(_updateState) Task=`5147145a-06dd-42c4-9ec6-d38fba7afad5`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:52:33,697::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:52:33,697::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`29c93bb0-148e-4bed-8f0d-6f1488daa67b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:52:33,698::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:52:33,698::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:52:33,698::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`29c93bb0-148e-4bed-8f0d-6f1488daa67b`::Granted request >Thread-338::DEBUG::2014-05-13 11:52:33,698::task::827::TaskManager.Task::(resourceAcquired) Task=`5147145a-06dd-42c4-9ec6-d38fba7afad5`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:52:33,699::task::990::TaskManager.Task::(_decref) Task=`5147145a-06dd-42c4-9ec6-d38fba7afad5`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:52:33,699::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:52:33,700::task::1185::TaskManager.Task::(prepare) Task=`5147145a-06dd-42c4-9ec6-d38fba7afad5`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:52:33,700::task::595::TaskManager.Task::(_updateState) Task=`5147145a-06dd-42c4-9ec6-d38fba7afad5`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:52:33,700::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:52:33,700::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:52:33,700::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:52:33,701::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:52:33,701::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:52:33,701::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:52:33,701::task::990::TaskManager.Task::(_decref) Task=`5147145a-06dd-42c4-9ec6-d38fba7afad5`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:52:34,501::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:52:35,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:52:35,297::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358355 s, 28.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:52:37,310::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:52:37,385::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323797 s, 31.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:52:37,521::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:52:39,398::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:52:39,474::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0335346 s, 30.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:52:40,539::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:52:41,462::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:52:41,470::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000590848 s, 6.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:52:41,486::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:52:41,565::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362585 s, 28.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:52:42,387::task::595::TaskManager.Task::(_updateState) Task=`85050936-64ee-48a9-aa94-d903e49b01f8`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:52:42,421::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:52:42,421::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000590848', 'lastCheck': '0.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:52:42,421::task::1185::TaskManager.Task::(prepare) Task=`85050936-64ee-48a9-aa94-d903e49b01f8`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000590848', 'lastCheck': '0.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:52:42,422::task::595::TaskManager.Task::(_updateState) Task=`85050936-64ee-48a9-aa94-d903e49b01f8`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:52:42,422::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:52:42,422::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:52:42,422::task::990::TaskManager.Task::(_decref) Task=`85050936-64ee-48a9-aa94-d903e49b01f8`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:52:43,559::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:52:43,578::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 11:52:43,719::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:52:43,720::task::595::TaskManager.Task::(_updateState) Task=`1139d57c-aaff-4f09-942f-c99ea8c9b8e2`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:52:43,720::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:52:43,720::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:52:43,721::task::1185::TaskManager.Task::(prepare) Task=`1139d57c-aaff-4f09-942f-c99ea8c9b8e2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:52:43,721::task::595::TaskManager.Task::(_updateState) Task=`1139d57c-aaff-4f09-942f-c99ea8c9b8e2`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:52:43,721::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:52:43,721::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:52:43,721::task::990::TaskManager.Task::(_decref) Task=`1139d57c-aaff-4f09-942f-c99ea8c9b8e2`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:52:43,953::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.331314 s, 3.1 MB/s\n'; <rc> = 0 >Thread-338::DEBUG::2014-05-13 11:52:43,997::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:52:43,997::task::595::TaskManager.Task::(_updateState) Task=`d0d5f369-e404-422e-aba4-60c04878c5ee`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:52:43,997::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:52:43,998::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`84023839-b2ff-45a6-a89e-a4b669f98294`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:52:43,998::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:52:43,998::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:52:43,998::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`84023839-b2ff-45a6-a89e-a4b669f98294`::Granted request >Thread-338::DEBUG::2014-05-13 11:52:43,999::task::827::TaskManager.Task::(resourceAcquired) Task=`d0d5f369-e404-422e-aba4-60c04878c5ee`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:52:43,999::task::990::TaskManager.Task::(_decref) Task=`d0d5f369-e404-422e-aba4-60c04878c5ee`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:52:44,000::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:52:44,000::task::1185::TaskManager.Task::(prepare) Task=`d0d5f369-e404-422e-aba4-60c04878c5ee`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:52:44,000::task::595::TaskManager.Task::(_updateState) Task=`d0d5f369-e404-422e-aba4-60c04878c5ee`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:52:44,000::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:52:44,000::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:52:44,001::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:52:44,001::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:52:44,001::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:52:44,001::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:52:44,001::task::990::TaskManager.Task::(_decref) Task=`d0d5f369-e404-422e-aba4-60c04878c5ee`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:52:45,966::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:52:46,043::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0339526 s, 30.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:52:46,577::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:52:48,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:52:48,137::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383688 s, 26.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:52:49,597::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:52:50,151::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:52:50,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356593 s, 28.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:52:51,476::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:52:51,484::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000753913 s, 5.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:52:52,242::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:52:52,324::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0395262 s, 25.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:52:52,622::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:52:54,023::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:52:54,024::task::595::TaskManager.Task::(_updateState) Task=`edf741be-a3d9-4b7a-a94c-0827bacccea6`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:52:54,024::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:52:54,024::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:52:54,025::task::1185::TaskManager.Task::(prepare) Task=`edf741be-a3d9-4b7a-a94c-0827bacccea6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:52:54,025::task::595::TaskManager.Task::(_updateState) Task=`edf741be-a3d9-4b7a-a94c-0827bacccea6`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:52:54,025::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:52:54,025::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:52:54,025::task::990::TaskManager.Task::(_decref) Task=`edf741be-a3d9-4b7a-a94c-0827bacccea6`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:52:54,030::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:52:54,031::task::595::TaskManager.Task::(_updateState) Task=`0c2791a4-c78f-4108-8038-67e08d443003`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:52:54,031::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:52:54,031::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9164d42f-120c-428c-9e69-a8f16c337344`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:52:54,032::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:52:54,032::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:52:54,032::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9164d42f-120c-428c-9e69-a8f16c337344`::Granted request >Thread-338::DEBUG::2014-05-13 11:52:54,032::task::827::TaskManager.Task::(resourceAcquired) Task=`0c2791a4-c78f-4108-8038-67e08d443003`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:52:54,032::task::990::TaskManager.Task::(_decref) Task=`0c2791a4-c78f-4108-8038-67e08d443003`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:52:54,033::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:52:54,033::task::1185::TaskManager.Task::(prepare) Task=`0c2791a4-c78f-4108-8038-67e08d443003`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:52:54,034::task::595::TaskManager.Task::(_updateState) Task=`0c2791a4-c78f-4108-8038-67e08d443003`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:52:54,034::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:52:54,034::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:52:54,034::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:52:54,034::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:52:54,035::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:52:54,035::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:52:54,035::task::990::TaskManager.Task::(_decref) Task=`0c2791a4-c78f-4108-8038-67e08d443003`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:52:54,337::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:52:54,419::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381769 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:52:55,640::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:52:56,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:52:56,829::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.358883 s, 2.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:52:57,562::task::595::TaskManager.Task::(_updateState) Task=`02ece90f-cb7e-4b66-a4ea-913d94519011`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:52:57,562::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:52:57,562::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000753913', 'lastCheck': '6.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:52:57,562::task::1185::TaskManager.Task::(prepare) Task=`02ece90f-cb7e-4b66-a4ea-913d94519011`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000753913', 'lastCheck': '6.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:52:57,563::task::595::TaskManager.Task::(_updateState) Task=`02ece90f-cb7e-4b66-a4ea-913d94519011`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:52:57,563::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:52:57,563::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:52:57,563::task::990::TaskManager.Task::(_decref) Task=`02ece90f-cb7e-4b66-a4ea-913d94519011`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:52:58,663::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:52:58,842::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:52:58,922::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0369967 s, 27.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:53:00,935::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:01,009::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310986 s, 32.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:53:01,491::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:53:01,499::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000504528 s, 8.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:53:01,682::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:53:03,022::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:03,102::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362091 s, 28.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:53:04,056::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:53:04,060::task::595::TaskManager.Task::(_updateState) Task=`ce52371c-d77b-461d-9386-71ad5a9da257`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:53:04,060::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:53:04,061::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:53:04,061::task::1185::TaskManager.Task::(prepare) Task=`ce52371c-d77b-461d-9386-71ad5a9da257`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:53:04,061::task::595::TaskManager.Task::(_updateState) Task=`ce52371c-d77b-461d-9386-71ad5a9da257`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:53:04,061::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:53:04,061::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:53:04,062::task::990::TaskManager.Task::(_decref) Task=`ce52371c-d77b-461d-9386-71ad5a9da257`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:53:04,066::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:53:04,067::task::595::TaskManager.Task::(_updateState) Task=`d0587735-d456-4281-9b03-2dd4726f8adf`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:53:04,067::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:53:04,067::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`279c3582-e4d1-4cfa-bdac-0c68fb6f8be7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:53:04,068::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:53:04,068::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:53:04,068::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`279c3582-e4d1-4cfa-bdac-0c68fb6f8be7`::Granted request >Thread-338::DEBUG::2014-05-13 11:53:04,068::task::827::TaskManager.Task::(resourceAcquired) Task=`d0587735-d456-4281-9b03-2dd4726f8adf`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:53:04,069::task::990::TaskManager.Task::(_decref) Task=`d0587735-d456-4281-9b03-2dd4726f8adf`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:53:04,069::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:53:04,069::task::1185::TaskManager.Task::(prepare) Task=`d0587735-d456-4281-9b03-2dd4726f8adf`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:53:04,070::task::595::TaskManager.Task::(_updateState) Task=`d0587735-d456-4281-9b03-2dd4726f8adf`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:53:04,070::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:53:04,070::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:53:04,070::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:53:04,070::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:53:04,071::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:53:04,071::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:53:04,071::task::990::TaskManager.Task::(_decref) Task=`d0587735-d456-4281-9b03-2dd4726f8adf`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:53:04,699::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:53:05,115::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:05,194::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359597 s, 28.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:53:07,207::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:07,282::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0330713 s, 31.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:53:07,719::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:53:09,295::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:09,390::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0506326 s, 20.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:53:10,740::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:53:11,403::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:11,478::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0327652 s, 31.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:53:11,506::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:53:11,514::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000722752 s, 5.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:53:12,692::task::595::TaskManager.Task::(_updateState) Task=`5f4cc8a3-550b-448f-b8a7-7837c2462bb3`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:53:12,692::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:53:12,692::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000722752', 'lastCheck': '1.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:53:12,693::task::1185::TaskManager.Task::(prepare) Task=`5f4cc8a3-550b-448f-b8a7-7837c2462bb3`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000722752', 'lastCheck': '1.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:53:12,693::task::595::TaskManager.Task::(_updateState) Task=`5f4cc8a3-550b-448f-b8a7-7837c2462bb3`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:53:12,693::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:53:12,693::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:53:12,693::task::990::TaskManager.Task::(_decref) Task=`5f4cc8a3-550b-448f-b8a7-7837c2462bb3`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:53:13,490::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:13,565::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321569 s, 31.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:53:13,760::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:53:14,090::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:53:14,091::task::595::TaskManager.Task::(_updateState) Task=`5faf0e8e-2bdd-4ac3-9513-7ad0bb2986d5`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:53:14,091::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:53:14,091::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:53:14,091::task::1185::TaskManager.Task::(prepare) Task=`5faf0e8e-2bdd-4ac3-9513-7ad0bb2986d5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:53:14,092::task::595::TaskManager.Task::(_updateState) Task=`5faf0e8e-2bdd-4ac3-9513-7ad0bb2986d5`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:53:14,092::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:53:14,092::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:53:14,092::task::990::TaskManager.Task::(_decref) Task=`5faf0e8e-2bdd-4ac3-9513-7ad0bb2986d5`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:53:14,097::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:53:14,097::task::595::TaskManager.Task::(_updateState) Task=`c5d3460e-29b1-4825-9ee2-07a0328eaafe`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:53:14,098::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:53:14,098::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`7b1d3443-a74a-4e90-80ab-ddbcd839be45`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:53:14,098::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:53:14,098::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:53:14,099::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`7b1d3443-a74a-4e90-80ab-ddbcd839be45`::Granted request >Thread-338::DEBUG::2014-05-13 11:53:14,099::task::827::TaskManager.Task::(resourceAcquired) Task=`c5d3460e-29b1-4825-9ee2-07a0328eaafe`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:53:14,099::task::990::TaskManager.Task::(_decref) Task=`c5d3460e-29b1-4825-9ee2-07a0328eaafe`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:53:14,100::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:53:14,100::task::1185::TaskManager.Task::(prepare) Task=`c5d3460e-29b1-4825-9ee2-07a0328eaafe`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:53:14,100::task::595::TaskManager.Task::(_updateState) Task=`c5d3460e-29b1-4825-9ee2-07a0328eaafe`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:53:14,100::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:53:14,101::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:53:14,101::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:53:14,101::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:53:14,101::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:53:14,101::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:53:14,102::task::990::TaskManager.Task::(_decref) Task=`c5d3460e-29b1-4825-9ee2-07a0328eaafe`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:53:15,578::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:15,655::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0324235 s, 31.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:53:16,778::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:53:17,667::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:17,747::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037825 s, 27.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:53:19,760::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:53:19,801::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:53:19,844::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0419647 s, 24.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:53:21,521::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:53:21,529::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000532127 s, 7.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:53:21,857::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:22,234::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.33573 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:53:22,828::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:53:24,122::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:53:24,122::task::595::TaskManager.Task::(_updateState) Task=`7c177b54-163a-41ac-ad34-d5a7f14f3cf1`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:53:24,122::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:53:24,123::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:53:24,123::task::1185::TaskManager.Task::(prepare) Task=`7c177b54-163a-41ac-ad34-d5a7f14f3cf1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:53:24,123::task::595::TaskManager.Task::(_updateState) Task=`7c177b54-163a-41ac-ad34-d5a7f14f3cf1`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:53:24,123::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:53:24,124::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:53:24,124::task::990::TaskManager.Task::(_decref) Task=`7c177b54-163a-41ac-ad34-d5a7f14f3cf1`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:53:24,128::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:53:24,128::task::595::TaskManager.Task::(_updateState) Task=`51b803e7-a321-45c5-8cb0-4ca56569f756`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:53:24,128::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:53:24,129::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f2550400-fcd1-4b51-9b8d-7a8abc4acc97`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:53:24,129::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:53:24,129::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:53:24,130::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f2550400-fcd1-4b51-9b8d-7a8abc4acc97`::Granted request >Thread-338::DEBUG::2014-05-13 11:53:24,130::task::827::TaskManager.Task::(resourceAcquired) Task=`51b803e7-a321-45c5-8cb0-4ca56569f756`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:53:24,130::task::990::TaskManager.Task::(_decref) Task=`51b803e7-a321-45c5-8cb0-4ca56569f756`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:53:24,131::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:53:24,131::task::1185::TaskManager.Task::(prepare) Task=`51b803e7-a321-45c5-8cb0-4ca56569f756`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:53:24,131::task::595::TaskManager.Task::(_updateState) Task=`51b803e7-a321-45c5-8cb0-4ca56569f756`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:53:24,131::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:53:24,132::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:53:24,132::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:53:24,132::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:53:24,132::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:53:24,132::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:53:24,133::task::990::TaskManager.Task::(_decref) Task=`51b803e7-a321-45c5-8cb0-4ca56569f756`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:53:24,247::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:24,328::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0386037 s, 26.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:53:25,847::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:53:26,341::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:26,417::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0331633 s, 30.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:53:27,828::task::595::TaskManager.Task::(_updateState) Task=`c26cd818-7eab-40c2-9791-0e2cd10ebede`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:53:27,828::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:53:27,828::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000532127', 'lastCheck': '6.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:53:27,828::task::1185::TaskManager.Task::(prepare) Task=`c26cd818-7eab-40c2-9791-0e2cd10ebede`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000532127', 'lastCheck': '6.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:53:27,828::task::595::TaskManager.Task::(_updateState) Task=`c26cd818-7eab-40c2-9791-0e2cd10ebede`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:53:27,829::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:53:27,829::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:53:27,829::task::990::TaskManager.Task::(_decref) Task=`c26cd818-7eab-40c2-9791-0e2cd10ebede`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:53:28,430::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:28,506::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0341509 s, 30.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:53:28,865::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:53:30,519::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:30,597::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0348841 s, 29.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:53:31,536::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:53:31,544::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000892178 s, 4.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:53:31,883::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:53:32,610::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:32,707::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0544078 s, 18.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:53:34,149::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:53:34,150::task::595::TaskManager.Task::(_updateState) Task=`b8fe6e5e-9016-45b5-b740-a622a23c1272`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:53:34,150::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:53:34,151::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:53:34,151::task::1185::TaskManager.Task::(prepare) Task=`b8fe6e5e-9016-45b5-b740-a622a23c1272`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:53:34,151::task::595::TaskManager.Task::(_updateState) Task=`b8fe6e5e-9016-45b5-b740-a622a23c1272`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:53:34,151::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:53:34,151::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:53:34,151::task::990::TaskManager.Task::(_decref) Task=`b8fe6e5e-9016-45b5-b740-a622a23c1272`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:53:34,156::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:53:34,156::task::595::TaskManager.Task::(_updateState) Task=`a19f043a-74e3-4267-8ae6-bf9e7ab6ab79`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:53:34,156::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:53:34,157::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`af1192f0-0450-4fe7-b7c2-1fc19b63c52a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:53:34,157::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:53:34,157::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:53:34,157::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`af1192f0-0450-4fe7-b7c2-1fc19b63c52a`::Granted request >Thread-338::DEBUG::2014-05-13 11:53:34,158::task::827::TaskManager.Task::(resourceAcquired) Task=`a19f043a-74e3-4267-8ae6-bf9e7ab6ab79`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:53:34,158::task::990::TaskManager.Task::(_decref) Task=`a19f043a-74e3-4267-8ae6-bf9e7ab6ab79`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:53:34,158::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:53:34,159::task::1185::TaskManager.Task::(prepare) Task=`a19f043a-74e3-4267-8ae6-bf9e7ab6ab79`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:53:34,159::task::595::TaskManager.Task::(_updateState) Task=`a19f043a-74e3-4267-8ae6-bf9e7ab6ab79`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:53:34,159::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:53:34,159::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:53:34,159::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:53:34,160::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:53:34,160::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:53:34,160::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:53:34,160::task::990::TaskManager.Task::(_decref) Task=`a19f043a-74e3-4267-8ae6-bf9e7ab6ab79`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:53:34,720::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:34,800::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355244 s, 28.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:53:34,902::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:53:36,813::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:36,900::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0449733 s, 22.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:53:37,921::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:53:38,913::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:38,991::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0366258 s, 28.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:53:40,940::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:53:41,004::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:41,080::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.033551 s, 30.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:53:41,551::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:53:41,559::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000665771 s, 6.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:53:42,967::task::595::TaskManager.Task::(_updateState) Task=`48f29328-eeb6-4af1-8c53-061dcafdde1e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:53:42,968::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:53:42,968::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000665771', 'lastCheck': '1.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:53:42,968::task::1185::TaskManager.Task::(prepare) Task=`48f29328-eeb6-4af1-8c53-061dcafdde1e`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000665771', 'lastCheck': '1.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:53:42,968::task::595::TaskManager.Task::(_updateState) Task=`48f29328-eeb6-4af1-8c53-061dcafdde1e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:53:42,968::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:53:42,968::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:53:42,969::task::990::TaskManager.Task::(_decref) Task=`48f29328-eeb6-4af1-8c53-061dcafdde1e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:53:43,093::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:43,168::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0334493 s, 30.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:53:43,962::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:53:44,180::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:53:44,180::task::595::TaskManager.Task::(_updateState) Task=`4a2ff3a1-9fa0-4bed-bc75-18f18d30d32f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:53:44,180::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:53:44,181::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:53:44,181::task::1185::TaskManager.Task::(prepare) Task=`4a2ff3a1-9fa0-4bed-bc75-18f18d30d32f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:53:44,181::task::595::TaskManager.Task::(_updateState) Task=`4a2ff3a1-9fa0-4bed-bc75-18f18d30d32f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:53:44,181::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:53:44,181::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:53:44,182::task::990::TaskManager.Task::(_decref) Task=`4a2ff3a1-9fa0-4bed-bc75-18f18d30d32f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:53:44,186::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:53:44,187::task::595::TaskManager.Task::(_updateState) Task=`28398cbd-0974-4c00-ad28-34b41f4a258e`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:53:44,187::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:53:44,187::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1ac8a2bc-e9b9-4508-9522-f1ee70ebab28`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:53:44,187::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:53:44,188::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:53:44,188::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1ac8a2bc-e9b9-4508-9522-f1ee70ebab28`::Granted request >Thread-338::DEBUG::2014-05-13 11:53:44,188::task::827::TaskManager.Task::(resourceAcquired) Task=`28398cbd-0974-4c00-ad28-34b41f4a258e`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:53:44,188::task::990::TaskManager.Task::(_decref) Task=`28398cbd-0974-4c00-ad28-34b41f4a258e`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:53:44,189::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:53:44,189::task::1185::TaskManager.Task::(prepare) Task=`28398cbd-0974-4c00-ad28-34b41f4a258e`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:53:44,190::task::595::TaskManager.Task::(_updateState) Task=`28398cbd-0974-4c00-ad28-34b41f4a258e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:53:44,190::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:53:44,190::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:53:44,190::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:53:44,190::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:53:44,190::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:53:44,191::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:53:44,191::task::990::TaskManager.Task::(_decref) Task=`28398cbd-0974-4c00-ad28-34b41f4a258e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:53:45,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:45,262::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382887 s, 26.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:53:46,980::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:53:47,275::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:47,348::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306084 s, 33.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:53:49,361::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:49,441::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0368998 s, 27.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:53:50,006::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:53:51,454::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:51,538::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0414203 s, 24.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:53:51,566::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:53:51,574::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000692447 s, 5.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:53:53,025::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:53:53,552::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:53,631::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035958 s, 28.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:53:54,211::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:53:54,212::task::595::TaskManager.Task::(_updateState) Task=`9b76a603-7a97-497e-b99b-d302cdd12990`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:53:54,212::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:53:54,213::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:53:54,213::task::1185::TaskManager.Task::(prepare) Task=`9b76a603-7a97-497e-b99b-d302cdd12990`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:53:54,213::task::595::TaskManager.Task::(_updateState) Task=`9b76a603-7a97-497e-b99b-d302cdd12990`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:53:54,213::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:53:54,213::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:53:54,214::task::990::TaskManager.Task::(_decref) Task=`9b76a603-7a97-497e-b99b-d302cdd12990`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:53:54,220::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:53:54,220::task::595::TaskManager.Task::(_updateState) Task=`95d42e7d-04fa-40a7-bece-dc1b5689c6b0`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:53:54,220::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:53:54,221::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b521a92d-0b61-4414-8349-5c1adb121ee2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:53:54,221::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:53:54,221::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:53:54,221::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b521a92d-0b61-4414-8349-5c1adb121ee2`::Granted request >Thread-338::DEBUG::2014-05-13 11:53:54,222::task::827::TaskManager.Task::(resourceAcquired) Task=`95d42e7d-04fa-40a7-bece-dc1b5689c6b0`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:53:54,222::task::990::TaskManager.Task::(_decref) Task=`95d42e7d-04fa-40a7-bece-dc1b5689c6b0`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:53:54,223::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:53:54,223::task::1185::TaskManager.Task::(prepare) Task=`95d42e7d-04fa-40a7-bece-dc1b5689c6b0`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:53:54,223::task::595::TaskManager.Task::(_updateState) Task=`95d42e7d-04fa-40a7-bece-dc1b5689c6b0`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:53:54,223::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:53:54,223::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:53:54,223::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:53:54,224::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:53:54,224::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:53:54,224::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:53:54,224::task::990::TaskManager.Task::(_decref) Task=`95d42e7d-04fa-40a7-bece-dc1b5689c6b0`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:53:55,644::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:55,719::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315672 s, 32.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:53:56,043::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:53:57,732::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:57,809::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0342997 s, 29.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:53:58,102::task::595::TaskManager.Task::(_updateState) Task=`023f52a9-1396-4c46-a198-f21bafbd7f23`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:53:58,102::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:53:58,103::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000692447', 'lastCheck': '6.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:53:58,103::task::1185::TaskManager.Task::(prepare) Task=`023f52a9-1396-4c46-a198-f21bafbd7f23`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000692447', 'lastCheck': '6.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:53:58,103::task::595::TaskManager.Task::(_updateState) Task=`023f52a9-1396-4c46-a198-f21bafbd7f23`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:53:58,103::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:53:58,103::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:53:58,104::task::990::TaskManager.Task::(_decref) Task=`023f52a9-1396-4c46-a198-f21bafbd7f23`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:53:59,061::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:53:59,822::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:53:59,903::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0380919 s, 26.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:54:01,581::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:54:01,588::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00064004 s, 6.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:54:01,916::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:54:01,995::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356097 s, 28.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:54:02,079::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:54:04,008::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:54:04,089::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364694 s, 28.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:54:04,242::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:54:04,243::task::595::TaskManager.Task::(_updateState) Task=`25e9c281-9c3e-46bd-9a5b-34ade7d7c98a`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:54:04,243::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:54:04,243::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:54:04,243::task::1185::TaskManager.Task::(prepare) Task=`25e9c281-9c3e-46bd-9a5b-34ade7d7c98a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:54:04,244::task::595::TaskManager.Task::(_updateState) Task=`25e9c281-9c3e-46bd-9a5b-34ade7d7c98a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:54:04,244::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:54:04,244::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:54:04,244::task::990::TaskManager.Task::(_decref) Task=`25e9c281-9c3e-46bd-9a5b-34ade7d7c98a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:54:04,249::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:54:04,249::task::595::TaskManager.Task::(_updateState) Task=`23ab7b87-5bc7-421d-b6ac-0af3e7de8038`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:54:04,249::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:54:04,250::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4ea23cd0-d548-4249-9819-109518e90f97`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:54:04,250::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:54:04,250::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:54:04,250::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4ea23cd0-d548-4249-9819-109518e90f97`::Granted request >Thread-338::DEBUG::2014-05-13 11:54:04,251::task::827::TaskManager.Task::(resourceAcquired) Task=`23ab7b87-5bc7-421d-b6ac-0af3e7de8038`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:54:04,251::task::990::TaskManager.Task::(_decref) Task=`23ab7b87-5bc7-421d-b6ac-0af3e7de8038`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:54:04,252::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:54:04,252::task::1185::TaskManager.Task::(prepare) Task=`23ab7b87-5bc7-421d-b6ac-0af3e7de8038`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:54:04,252::task::595::TaskManager.Task::(_updateState) Task=`23ab7b87-5bc7-421d-b6ac-0af3e7de8038`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:54:04,252::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:54:04,252::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:54:04,253::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:54:04,253::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:54:04,253::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:54:04,253::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:54:04,253::task::990::TaskManager.Task::(_decref) Task=`23ab7b87-5bc7-421d-b6ac-0af3e7de8038`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:54:05,097::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:54:06,102::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:54:06,181::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365637 s, 28.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:54:08,116::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:54:08,194::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:54:08,269::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311278 s, 32.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:54:10,282::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:54:10,374::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0493344 s, 20.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:54:11,135::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:54:11,596::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:54:11,603::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000530269 s, 7.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:54:12,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:54:12,467::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0391344 s, 26.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:54:13,239::task::595::TaskManager.Task::(_updateState) Task=`730e188b-27aa-45e1-be00-6a27a82f37fa`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:54:13,239::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:54:13,239::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000530269', 'lastCheck': '1.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:54:13,240::task::1185::TaskManager.Task::(prepare) Task=`730e188b-27aa-45e1-be00-6a27a82f37fa`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000530269', 'lastCheck': '1.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:54:13,240::task::595::TaskManager.Task::(_updateState) Task=`730e188b-27aa-45e1-be00-6a27a82f37fa`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:54:13,240::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:54:13,240::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:54:13,240::task::990::TaskManager.Task::(_decref) Task=`730e188b-27aa-45e1-be00-6a27a82f37fa`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:54:14,154::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:54:14,272::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:54:14,273::task::595::TaskManager.Task::(_updateState) Task=`68bd8e31-3838-438c-ba11-99c9df24aa84`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:54:14,273::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:54:14,273::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:54:14,273::task::1185::TaskManager.Task::(prepare) Task=`68bd8e31-3838-438c-ba11-99c9df24aa84`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:54:14,274::task::595::TaskManager.Task::(_updateState) Task=`68bd8e31-3838-438c-ba11-99c9df24aa84`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:54:14,274::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:54:14,274::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:54:14,274::task::990::TaskManager.Task::(_decref) Task=`68bd8e31-3838-438c-ba11-99c9df24aa84`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:54:14,279::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:54:14,279::task::595::TaskManager.Task::(_updateState) Task=`cb19a8a0-5d7c-4e74-8130-906cbf1c7393`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:54:14,279::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:54:14,280::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`7c52805b-aaf7-46e5-84b3-650f74778b19`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:54:14,280::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:54:14,280::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:54:14,280::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`7c52805b-aaf7-46e5-84b3-650f74778b19`::Granted request >Thread-338::DEBUG::2014-05-13 11:54:14,280::task::827::TaskManager.Task::(resourceAcquired) Task=`cb19a8a0-5d7c-4e74-8130-906cbf1c7393`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:54:14,281::task::990::TaskManager.Task::(_decref) Task=`cb19a8a0-5d7c-4e74-8130-906cbf1c7393`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:54:14,281::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:54:14,282::task::1185::TaskManager.Task::(prepare) Task=`cb19a8a0-5d7c-4e74-8130-906cbf1c7393`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:54:14,282::task::595::TaskManager.Task::(_updateState) Task=`cb19a8a0-5d7c-4e74-8130-906cbf1c7393`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:54:14,282::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:54:14,282::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:54:14,282::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:54:14,283::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:54:14,283::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:54:14,283::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:54:14,283::task::990::TaskManager.Task::(_decref) Task=`cb19a8a0-5d7c-4e74-8130-906cbf1c7393`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:54:14,480::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:54:14,561::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0385111 s, 26.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:54:16,574::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:54:16,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0574335 s, 17.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:54:17,172::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:54:18,686::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:54:18,780::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0342683 s, 29.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:54:20,190::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:54:20,794::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:54:20,874::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.038253 s, 26.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:54:21,610::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:54:21,617::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000676096 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:54:22,887::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:54:23,209::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:54:23,263::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.332945 s, 3.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:54:24,301::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:54:24,302::task::595::TaskManager.Task::(_updateState) Task=`71c2a9df-e077-4bc2-b65b-88ebd62e3897`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:54:24,302::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:54:24,302::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:54:24,302::task::1185::TaskManager.Task::(prepare) Task=`71c2a9df-e077-4bc2-b65b-88ebd62e3897`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:54:24,303::task::595::TaskManager.Task::(_updateState) Task=`71c2a9df-e077-4bc2-b65b-88ebd62e3897`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:54:24,303::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:54:24,303::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:54:24,303::task::990::TaskManager.Task::(_decref) Task=`71c2a9df-e077-4bc2-b65b-88ebd62e3897`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:54:24,308::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:54:24,308::task::595::TaskManager.Task::(_updateState) Task=`9a278325-da98-4eac-998b-72ebd235e9ed`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:54:24,308::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:54:24,309::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6cdba39a-c2d3-4508-8c9c-cc6b2778961d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:54:24,309::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:54:24,309::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:54:24,309::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6cdba39a-c2d3-4508-8c9c-cc6b2778961d`::Granted request >Thread-338::DEBUG::2014-05-13 11:54:24,310::task::827::TaskManager.Task::(resourceAcquired) Task=`9a278325-da98-4eac-998b-72ebd235e9ed`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:54:24,310::task::990::TaskManager.Task::(_decref) Task=`9a278325-da98-4eac-998b-72ebd235e9ed`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:54:24,311::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:54:24,311::task::1185::TaskManager.Task::(prepare) Task=`9a278325-da98-4eac-998b-72ebd235e9ed`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:54:24,311::task::595::TaskManager.Task::(_updateState) Task=`9a278325-da98-4eac-998b-72ebd235e9ed`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:54:24,311::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:54:24,311::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:54:24,312::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:54:24,312::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:54:24,312::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:54:24,312::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:54:24,312::task::990::TaskManager.Task::(_decref) Task=`9a278325-da98-4eac-998b-72ebd235e9ed`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:54:25,276::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:54:25,351::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0331487 s, 30.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:54:26,235::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:54:27,364::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:54:27,446::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382662 s, 26.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:54:28,376::task::595::TaskManager.Task::(_updateState) Task=`05e9b9c6-c7e3-4d25-ac1f-8ca6994ec906`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:54:28,377::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:54:28,377::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000676096', 'lastCheck': '6.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:54:28,377::task::1185::TaskManager.Task::(prepare) Task=`05e9b9c6-c7e3-4d25-ac1f-8ca6994ec906`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000676096', 'lastCheck': '6.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:54:28,377::task::595::TaskManager.Task::(_updateState) Task=`05e9b9c6-c7e3-4d25-ac1f-8ca6994ec906`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:54:28,377::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:54:28,377::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:54:28,378::task::990::TaskManager.Task::(_decref) Task=`05e9b9c6-c7e3-4d25-ac1f-8ca6994ec906`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:54:29,254::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:54:29,459::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:54:29,539::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.038212 s, 26.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:54:31,551::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 11:54:31,644::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:54:31,663::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000583312 s, 7.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:54:31,690::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353683 s, 29.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:54:32,271::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:54:33,703::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:54:33,779::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0330349 s, 31.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:54:34,333::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:54:34,333::task::595::TaskManager.Task::(_updateState) Task=`e6bb073d-6beb-4669-89da-ed8ecab8849f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:54:34,334::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:54:34,334::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:54:34,334::task::1185::TaskManager.Task::(prepare) Task=`e6bb073d-6beb-4669-89da-ed8ecab8849f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:54:34,334::task::595::TaskManager.Task::(_updateState) Task=`e6bb073d-6beb-4669-89da-ed8ecab8849f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:54:34,334::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:54:34,335::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:54:34,335::task::990::TaskManager.Task::(_decref) Task=`e6bb073d-6beb-4669-89da-ed8ecab8849f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:54:34,339::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:54:34,340::task::595::TaskManager.Task::(_updateState) Task=`149397c1-192d-4eca-950f-f5d7457ae42a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:54:34,340::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:54:34,340::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`7cb89633-6e18-45b0-b71b-229d38483b15`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:54:34,341::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:54:34,341::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:54:34,341::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`7cb89633-6e18-45b0-b71b-229d38483b15`::Granted request >Thread-338::DEBUG::2014-05-13 11:54:34,341::task::827::TaskManager.Task::(resourceAcquired) Task=`149397c1-192d-4eca-950f-f5d7457ae42a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:54:34,342::task::990::TaskManager.Task::(_decref) Task=`149397c1-192d-4eca-950f-f5d7457ae42a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:54:34,342::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:54:34,343::task::1185::TaskManager.Task::(prepare) Task=`149397c1-192d-4eca-950f-f5d7457ae42a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:54:34,343::task::595::TaskManager.Task::(_updateState) Task=`149397c1-192d-4eca-950f-f5d7457ae42a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:54:34,343::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:54:34,343::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:54:34,343::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:54:34,344::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:54:34,344::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:54:34,344::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:54:34,344::task::990::TaskManager.Task::(_decref) Task=`149397c1-192d-4eca-950f-f5d7457ae42a`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:54:35,291::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:54:35,792::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:54:35,872::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0377024 s, 27.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:54:37,885::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:54:37,969::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0417351 s, 24.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:54:38,313::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:54:39,982::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:54:40,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311503 s, 32.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:54:41,338::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:54:41,690::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:54:41,697::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000683472 s, 6.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:54:42,069::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:54:42,150::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0366228 s, 28.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:54:43,510::task::595::TaskManager.Task::(_updateState) Task=`cfe47005-b455-44ce-965d-e7a3bbe6ad2d`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:54:43,511::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:54:43,511::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000683472', 'lastCheck': '1.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:54:43,511::task::1185::TaskManager.Task::(prepare) Task=`cfe47005-b455-44ce-965d-e7a3bbe6ad2d`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000683472', 'lastCheck': '1.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:54:43,511::task::595::TaskManager.Task::(_updateState) Task=`cfe47005-b455-44ce-965d-e7a3bbe6ad2d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:54:43,511::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:54:43,512::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:54:43,512::task::990::TaskManager.Task::(_decref) Task=`cfe47005-b455-44ce-965d-e7a3bbe6ad2d`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:54:44,164::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:54:44,242::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356878 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:54:44,359::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:54:44,364::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [30b3ccf] >Thread-13::DEBUG::2014-05-13 11:54:44,364::task::595::TaskManager.Task::(_updateState) Task=`64276c09-8eb9-474c-986f-dbf442257d6e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:54:44,365::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:54:44,365::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:54:44,365::task::1185::TaskManager.Task::(prepare) Task=`64276c09-8eb9-474c-986f-dbf442257d6e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:54:44,365::task::595::TaskManager.Task::(_updateState) Task=`64276c09-8eb9-474c-986f-dbf442257d6e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:54:44,365::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:54:44,366::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:54:44,366::task::990::TaskManager.Task::(_decref) Task=`64276c09-8eb9-474c-986f-dbf442257d6e`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:54:44,374::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [30b3ccf] >Thread-338::DEBUG::2014-05-13 11:54:44,374::task::595::TaskManager.Task::(_updateState) Task=`6009801d-73fe-40b5-a2e6-a5d5918e54d9`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:54:44,374::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:54:44,375::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`01507bcb-4722-425c-81c7-f46593f2185c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:54:44,375::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:54:44,375::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:54:44,375::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`01507bcb-4722-425c-81c7-f46593f2185c`::Granted request >Thread-338::DEBUG::2014-05-13 11:54:44,376::task::827::TaskManager.Task::(resourceAcquired) Task=`6009801d-73fe-40b5-a2e6-a5d5918e54d9`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:54:44,376::task::990::TaskManager.Task::(_decref) Task=`6009801d-73fe-40b5-a2e6-a5d5918e54d9`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:54:44,376::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:54:44,377::task::1185::TaskManager.Task::(prepare) Task=`6009801d-73fe-40b5-a2e6-a5d5918e54d9`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:54:44,377::task::595::TaskManager.Task::(_updateState) Task=`6009801d-73fe-40b5-a2e6-a5d5918e54d9`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:54:44,377::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:54:44,377::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:54:44,378::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:54:44,378::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:54:44,378::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:54:44,378::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:54:44,379::task::990::TaskManager.Task::(_decref) Task=`6009801d-73fe-40b5-a2e6-a5d5918e54d9`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:54:46,255::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:54:46,331::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0335035 s, 30.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:54:47,381::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:54:48,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:54:48,426::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0388555 s, 26.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:54:50,398::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:54:50,439::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:54:50,808::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.327288 s, 3.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:54:51,704::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:54:51,712::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000599261 s, 6.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:54:52,821::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:54:52,914::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0502677 s, 20.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:54:53,418::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:54:54,396::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:54:54,397::task::595::TaskManager.Task::(_updateState) Task=`76611715-de89-4c2d-b542-907cc5b80c68`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:54:54,397::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:54:54,398::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:54:54,398::task::1185::TaskManager.Task::(prepare) Task=`76611715-de89-4c2d-b542-907cc5b80c68`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:54:54,398::task::595::TaskManager.Task::(_updateState) Task=`76611715-de89-4c2d-b542-907cc5b80c68`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:54:54,398::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:54:54,398::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:54:54,398::task::990::TaskManager.Task::(_decref) Task=`76611715-de89-4c2d-b542-907cc5b80c68`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:54:54,403::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:54:54,403::task::595::TaskManager.Task::(_updateState) Task=`472f41d5-13dc-40e0-bb51-eddd546eae76`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:54:54,404::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:54:54,404::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`05a6de73-9c45-4400-ac8e-1a1d5470e6f0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:54:54,404::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:54:54,404::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:54:54,405::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`05a6de73-9c45-4400-ac8e-1a1d5470e6f0`::Granted request >Thread-338::DEBUG::2014-05-13 11:54:54,405::task::827::TaskManager.Task::(resourceAcquired) Task=`472f41d5-13dc-40e0-bb51-eddd546eae76`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:54:54,405::task::990::TaskManager.Task::(_decref) Task=`472f41d5-13dc-40e0-bb51-eddd546eae76`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:54:54,406::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:54:54,406::task::1185::TaskManager.Task::(prepare) Task=`472f41d5-13dc-40e0-bb51-eddd546eae76`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:54:54,406::task::595::TaskManager.Task::(_updateState) Task=`472f41d5-13dc-40e0-bb51-eddd546eae76`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:54:54,406::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:54:54,407::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:54:54,407::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:54:54,407::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:54:54,407::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:54:54,407::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:54:54,408::task::990::TaskManager.Task::(_decref) Task=`472f41d5-13dc-40e0-bb51-eddd546eae76`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:54:54,927::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:54:55,001::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316971 s, 32.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:54:56,437::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:54:57,015::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:54:57,093::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356311 s, 28.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:54:58,652::task::595::TaskManager.Task::(_updateState) Task=`c7031408-0c29-4fe0-afe4-0561fbef0c64`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:54:58,652::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:54:58,653::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000599261', 'lastCheck': '6.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:54:58,653::task::1185::TaskManager.Task::(prepare) Task=`c7031408-0c29-4fe0-afe4-0561fbef0c64`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000599261', 'lastCheck': '6.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:54:58,653::task::595::TaskManager.Task::(_updateState) Task=`c7031408-0c29-4fe0-afe4-0561fbef0c64`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:54:58,653::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:54:58,653::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:54:58,653::task::990::TaskManager.Task::(_decref) Task=`c7031408-0c29-4fe0-afe4-0561fbef0c64`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:54:59,106::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:54:59,185::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363589 s, 28.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:54:59,456::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:55:01,197::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:55:01,272::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0326818 s, 31.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:55:01,717::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 41830dad-f012-4b4d-b636-863f66d5ca4b >Thread-322::DEBUG::2014-05-13 11:55:01,717::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-322::DEBUG::2014-05-13 11:55:01,720::blockSD::336::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:55:01,728::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.00062243 s, 3.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:55:01,728::misc::287::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.00062243 s, 3.3 MB/s'], size: 2048 >Thread-322::DEBUG::2014-05-13 11:55:01,728::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-322::WARNING::2014-05-13 11:55:01,728::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-322::DEBUG::2014-05-13 11:55:01,729::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-322::DEBUG::2014-05-13 11:55:01,729::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:55:01,729::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:55:01,730::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:55:01,730::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:55:01,730::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 11:55:01,731::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 11:55:02,003::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:55:02,005::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 11:55:02,005::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'LOCKPOLICY=', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', 'DESCRIPTION=ISCSI', 'LOGBLKSIZE=512', 'TYPE=ISCSI', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'PHYBLKSIZE=512', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', 'CLASS=Data', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'POOL_DESCRIPTION=DC_2', 'POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'MASTER_VERSION=1', 'ROLE=Master', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', '_SHA_CKSUM=28bb2d7cc60e0d779eb69fe93d050c16c769f069', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0'] >Thread-322::INFO::2014-05-13 11:55:02,006::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_imageNS already registered >Thread-322::INFO::2014-05-13 11:55:02,007::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_volumeNS already registered >Thread-322::INFO::2014-05-13 11:55:02,007::blockSD::452::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_lvmActivationNS already registered >Thread-322::DEBUG::2014-05-13 11:55:02,008::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 11:55:02,220::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:55:02,223::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:55:02,232::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000587214 s, 7.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:55:02,475::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:55:03,285::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:55:03,377::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0491189 s, 20.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:55:04,425::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1c80df6] >Thread-13::DEBUG::2014-05-13 11:55:04,425::task::595::TaskManager.Task::(_updateState) Task=`ad638685-dbc8-432d-a2a5-6f36d50f419f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:55:04,426::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:55:04,426::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:55:04,426::task::1185::TaskManager.Task::(prepare) Task=`ad638685-dbc8-432d-a2a5-6f36d50f419f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:55:04,426::task::595::TaskManager.Task::(_updateState) Task=`ad638685-dbc8-432d-a2a5-6f36d50f419f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:55:04,426::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:55:04,427::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:55:04,427::task::990::TaskManager.Task::(_decref) Task=`ad638685-dbc8-432d-a2a5-6f36d50f419f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:55:04,432::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1c80df6] >Thread-338::DEBUG::2014-05-13 11:55:04,432::task::595::TaskManager.Task::(_updateState) Task=`4b4e9000-e2dd-4a3d-a0b5-01e40317018a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:55:04,432::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:55:04,433::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3aa6d45b-92fc-4a54-9053-abc602dd94ed`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:55:04,433::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:55:04,433::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:55:04,433::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3aa6d45b-92fc-4a54-9053-abc602dd94ed`::Granted request >Thread-338::DEBUG::2014-05-13 11:55:04,434::task::827::TaskManager.Task::(resourceAcquired) Task=`4b4e9000-e2dd-4a3d-a0b5-01e40317018a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:55:04,434::task::990::TaskManager.Task::(_decref) Task=`4b4e9000-e2dd-4a3d-a0b5-01e40317018a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:55:04,434::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:55:04,435::task::1185::TaskManager.Task::(prepare) Task=`4b4e9000-e2dd-4a3d-a0b5-01e40317018a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:55:04,435::task::595::TaskManager.Task::(_updateState) Task=`4b4e9000-e2dd-4a3d-a0b5-01e40317018a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:55:04,435::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:55:04,435::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:55:04,435::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:55:04,436::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:55:04,436::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:55:04,436::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:55:04,436::task::990::TaskManager.Task::(_decref) Task=`4b4e9000-e2dd-4a3d-a0b5-01e40317018a`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:55:05,390::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:55:05,469::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364222 s, 28.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:55:05,493::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:55:07,482::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:55:07,557::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0333702 s, 30.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:55:08,512::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:55:09,570::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:55:09,646::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319021 s, 32.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:55:11,537::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:55:11,659::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:55:11,739::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0367702 s, 27.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:55:12,240::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:55:12,248::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000780013 s, 5.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:55:13,752::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 11:55:13,790::task::595::TaskManager.Task::(_updateState) Task=`0d3b3d9b-fe2d-48d8-afea-4215952d1c81`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:55:13,791::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:55:13,791::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000780013', 'lastCheck': '1.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:55:13,792::task::1185::TaskManager.Task::(prepare) Task=`0d3b3d9b-fe2d-48d8-afea-4215952d1c81`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000780013', 'lastCheck': '1.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:55:13,792::task::595::TaskManager.Task::(_updateState) Task=`0d3b3d9b-fe2d-48d8-afea-4215952d1c81`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:55:13,792::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:55:13,793::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:55:13,793::task::990::TaskManager.Task::(_decref) Task=`0d3b3d9b-fe2d-48d8-afea-4215952d1c81`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:55:13,841::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0415777 s, 24.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:55:14,455::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:55:14,455::task::595::TaskManager.Task::(_updateState) Task=`ea4aad15-870d-464f-ada5-cad0bc71c1a9`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:55:14,456::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:55:14,456::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:55:14,456::task::1185::TaskManager.Task::(prepare) Task=`ea4aad15-870d-464f-ada5-cad0bc71c1a9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:55:14,456::task::595::TaskManager.Task::(_updateState) Task=`ea4aad15-870d-464f-ada5-cad0bc71c1a9`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:55:14,456::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:55:14,457::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:55:14,457::task::990::TaskManager.Task::(_decref) Task=`ea4aad15-870d-464f-ada5-cad0bc71c1a9`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:55:14,461::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:55:14,462::task::595::TaskManager.Task::(_updateState) Task=`742c2ada-1927-4e25-a361-2fcce0ea7f77`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:55:14,462::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:55:14,462::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`bf872fbd-e1fb-44c0-9778-9167c9252d7f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:55:14,463::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:55:14,463::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:55:14,463::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`bf872fbd-e1fb-44c0-9778-9167c9252d7f`::Granted request >Thread-338::DEBUG::2014-05-13 11:55:14,463::task::827::TaskManager.Task::(resourceAcquired) Task=`742c2ada-1927-4e25-a361-2fcce0ea7f77`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:55:14,464::task::990::TaskManager.Task::(_decref) Task=`742c2ada-1927-4e25-a361-2fcce0ea7f77`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:55:14,464::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:55:14,465::task::1185::TaskManager.Task::(prepare) Task=`742c2ada-1927-4e25-a361-2fcce0ea7f77`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:55:14,465::task::595::TaskManager.Task::(_updateState) Task=`742c2ada-1927-4e25-a361-2fcce0ea7f77`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:55:14,465::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:55:14,465::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:55:14,465::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:55:14,466::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:55:14,466::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:55:14,466::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:55:14,466::task::990::TaskManager.Task::(_decref) Task=`742c2ada-1927-4e25-a361-2fcce0ea7f77`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:55:14,556::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:55:15,853::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:55:15,930::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.033669 s, 30.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:55:17,575::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:55:17,943::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:55:18,033::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0484357 s, 21.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:55:20,046::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:55:20,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382083 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:55:20,594::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:55:22,139::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:55:22,214::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031761 s, 32.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:55:22,255::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:55:22,265::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00253719 s, 1.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:55:23,616::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:55:24,227::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:55:24,316::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0459841 s, 22.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:55:24,490::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:55:24,491::task::595::TaskManager.Task::(_updateState) Task=`d2ec927c-31e4-478d-8cd1-adc6b74cc662`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:55:24,491::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:55:24,491::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:55:24,491::task::1185::TaskManager.Task::(prepare) Task=`d2ec927c-31e4-478d-8cd1-adc6b74cc662`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:55:24,492::task::595::TaskManager.Task::(_updateState) Task=`d2ec927c-31e4-478d-8cd1-adc6b74cc662`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:55:24,492::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:55:24,492::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:55:24,492::task::990::TaskManager.Task::(_decref) Task=`d2ec927c-31e4-478d-8cd1-adc6b74cc662`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:55:24,497::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:55:24,497::task::595::TaskManager.Task::(_updateState) Task=`54bf0a48-f64b-4c67-ad02-77de4114cd5a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:55:24,497::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:55:24,498::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4dca90be-3091-409f-9b80-fb0d1b88a54e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:55:24,498::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:55:24,498::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:55:24,498::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4dca90be-3091-409f-9b80-fb0d1b88a54e`::Granted request >Thread-338::DEBUG::2014-05-13 11:55:24,499::task::827::TaskManager.Task::(resourceAcquired) Task=`54bf0a48-f64b-4c67-ad02-77de4114cd5a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:55:24,499::task::990::TaskManager.Task::(_decref) Task=`54bf0a48-f64b-4c67-ad02-77de4114cd5a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:55:24,499::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:55:24,500::task::1185::TaskManager.Task::(prepare) Task=`54bf0a48-f64b-4c67-ad02-77de4114cd5a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:55:24,500::task::595::TaskManager.Task::(_updateState) Task=`54bf0a48-f64b-4c67-ad02-77de4114cd5a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:55:24,500::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:55:24,500::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:55:24,500::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:55:24,501::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:55:24,501::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:55:24,501::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:55:24,501::task::990::TaskManager.Task::(_decref) Task=`54bf0a48-f64b-4c67-ad02-77de4114cd5a`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:55:26,329::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:55:26,403::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031356 s, 32.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:55:26,635::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:55:28,416::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:55:28,491::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310575 s, 33.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:55:28,930::task::595::TaskManager.Task::(_updateState) Task=`3ececa9c-eb91-49ea-ad41-253a60439657`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:55:28,931::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:55:28,931::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00253719', 'lastCheck': '6.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:55:28,931::task::1185::TaskManager.Task::(prepare) Task=`3ececa9c-eb91-49ea-ad41-253a60439657`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00253719', 'lastCheck': '6.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:55:28,931::task::595::TaskManager.Task::(_updateState) Task=`3ececa9c-eb91-49ea-ad41-253a60439657`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:55:28,931::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:55:28,932::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:55:28,932::task::990::TaskManager.Task::(_decref) Task=`3ececa9c-eb91-49ea-ad41-253a60439657`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:55:29,653::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:55:30,504::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:55:30,859::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.312104 s, 3.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:55:32,272::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:55:32,280::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000729729 s, 5.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:55:32,672::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:55:32,872::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:55:32,953::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379155 s, 27.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:55:34,519::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:55:34,519::task::595::TaskManager.Task::(_updateState) Task=`821c6b72-4853-480f-818f-dcd693c96ed1`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:55:34,520::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:55:34,520::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:55:34,520::task::1185::TaskManager.Task::(prepare) Task=`821c6b72-4853-480f-818f-dcd693c96ed1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:55:34,520::task::595::TaskManager.Task::(_updateState) Task=`821c6b72-4853-480f-818f-dcd693c96ed1`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:55:34,521::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:55:34,521::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:55:34,521::task::990::TaskManager.Task::(_decref) Task=`821c6b72-4853-480f-818f-dcd693c96ed1`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:55:34,525::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:55:34,525::task::595::TaskManager.Task::(_updateState) Task=`6159a6fb-9423-4fe3-8f2c-ab116fd33881`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:55:34,526::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:55:34,526::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`65ea5302-b105-4edb-a8a7-e80eea084e68`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:55:34,526::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:55:34,527::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:55:34,528::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`65ea5302-b105-4edb-a8a7-e80eea084e68`::Granted request >Thread-338::DEBUG::2014-05-13 11:55:34,528::task::827::TaskManager.Task::(resourceAcquired) Task=`6159a6fb-9423-4fe3-8f2c-ab116fd33881`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:55:34,528::task::990::TaskManager.Task::(_decref) Task=`6159a6fb-9423-4fe3-8f2c-ab116fd33881`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:55:34,529::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:55:34,529::task::1185::TaskManager.Task::(prepare) Task=`6159a6fb-9423-4fe3-8f2c-ab116fd33881`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:55:34,529::task::595::TaskManager.Task::(_updateState) Task=`6159a6fb-9423-4fe3-8f2c-ab116fd33881`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:55:34,529::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:55:34,530::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:55:34,530::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:55:34,530::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:55:34,530::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:55:34,530::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:55:34,531::task::990::TaskManager.Task::(_decref) Task=`6159a6fb-9423-4fe3-8f2c-ab116fd33881`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:55:34,973::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:55:35,058::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374597 s, 27.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:55:35,693::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:55:37,071::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:55:37,147::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319402 s, 32.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:55:38,711::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:55:39,160::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:55:39,239::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361791 s, 28.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:55:41,252::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:55:41,327::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0330536 s, 31.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:55:41,735::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:55:42,287::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:55:42,295::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000648561 s, 6.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:55:43,340::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:55:43,428::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0462468 s, 22.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:55:44,075::task::595::TaskManager.Task::(_updateState) Task=`022156ca-fb2e-4c71-bfa3-6314ff59d16b`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:55:44,075::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:55:44,075::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000648561', 'lastCheck': '1.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:55:44,075::task::1185::TaskManager.Task::(prepare) Task=`022156ca-fb2e-4c71-bfa3-6314ff59d16b`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000648561', 'lastCheck': '1.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:55:44,075::task::595::TaskManager.Task::(_updateState) Task=`022156ca-fb2e-4c71-bfa3-6314ff59d16b`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:55:44,076::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:55:44,076::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:55:44,076::task::990::TaskManager.Task::(_decref) Task=`022156ca-fb2e-4c71-bfa3-6314ff59d16b`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 11:55:44,548::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:55:44,548::task::595::TaskManager.Task::(_updateState) Task=`bf2f3db2-852c-4f79-9b74-60a95760e6e0`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:55:44,548::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:55:44,549::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:55:44,549::task::1185::TaskManager.Task::(prepare) Task=`bf2f3db2-852c-4f79-9b74-60a95760e6e0`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:55:44,549::task::595::TaskManager.Task::(_updateState) Task=`bf2f3db2-852c-4f79-9b74-60a95760e6e0`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:55:44,549::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:55:44,549::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:55:44,550::task::990::TaskManager.Task::(_decref) Task=`bf2f3db2-852c-4f79-9b74-60a95760e6e0`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:55:44,555::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:55:44,555::task::595::TaskManager.Task::(_updateState) Task=`70a3fe79-6e22-47b5-a161-e90fe8763b36`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:55:44,555::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:55:44,556::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4e7080fb-ee4c-4c79-bb76-86351a168164`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:55:44,556::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:55:44,556::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:55:44,556::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4e7080fb-ee4c-4c79-bb76-86351a168164`::Granted request >Thread-338::DEBUG::2014-05-13 11:55:44,557::task::827::TaskManager.Task::(resourceAcquired) Task=`70a3fe79-6e22-47b5-a161-e90fe8763b36`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:55:44,557::task::990::TaskManager.Task::(_decref) Task=`70a3fe79-6e22-47b5-a161-e90fe8763b36`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:55:44,558::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:55:44,558::task::1185::TaskManager.Task::(prepare) Task=`70a3fe79-6e22-47b5-a161-e90fe8763b36`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:55:44,558::task::595::TaskManager.Task::(_updateState) Task=`70a3fe79-6e22-47b5-a161-e90fe8763b36`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:55:44,558::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:55:44,558::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:55:44,559::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:55:44,559::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:55:44,559::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:55:44,559::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:55:44,559::task::990::TaskManager.Task::(_decref) Task=`70a3fe79-6e22-47b5-a161-e90fe8763b36`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:55:44,764::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:55:45,441::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:55:45,519::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358923 s, 28.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:55:47,532::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:55:47,608::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318128 s, 32.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:55:47,783::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:55:49,621::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:55:49,700::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0376578 s, 27.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:55:50,805::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:55:51,713::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:55:51,789::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311708 s, 32.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:55:52,341::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:55:52,349::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000790363 s, 5.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:55:53,802::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:55:53,825::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:55:53,886::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0420077 s, 24.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:55:54,577::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2555e502] >Thread-13::DEBUG::2014-05-13 11:55:54,578::task::595::TaskManager.Task::(_updateState) Task=`c23e3174-e465-4f8a-9feb-e40e16bd9a48`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:55:54,578::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:55:54,578::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:55:54,578::task::1185::TaskManager.Task::(prepare) Task=`c23e3174-e465-4f8a-9feb-e40e16bd9a48`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:55:54,579::task::595::TaskManager.Task::(_updateState) Task=`c23e3174-e465-4f8a-9feb-e40e16bd9a48`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:55:54,579::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:55:54,579::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:55:54,579::task::990::TaskManager.Task::(_decref) Task=`c23e3174-e465-4f8a-9feb-e40e16bd9a48`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:55:54,584::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2555e502] >Thread-338::DEBUG::2014-05-13 11:55:54,584::task::595::TaskManager.Task::(_updateState) Task=`8e38d69c-42a0-49ca-8932-ad381cceb23b`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:55:54,584::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:55:54,584::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`466e3e5f-9af2-4959-80e6-330aed85852c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:55:54,585::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:55:54,585::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:55:54,585::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`466e3e5f-9af2-4959-80e6-330aed85852c`::Granted request >Thread-338::DEBUG::2014-05-13 11:55:54,585::task::827::TaskManager.Task::(resourceAcquired) Task=`8e38d69c-42a0-49ca-8932-ad381cceb23b`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:55:54,585::task::990::TaskManager.Task::(_decref) Task=`8e38d69c-42a0-49ca-8932-ad381cceb23b`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:55:54,586::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:55:54,586::task::1185::TaskManager.Task::(prepare) Task=`8e38d69c-42a0-49ca-8932-ad381cceb23b`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:55:54,586::task::595::TaskManager.Task::(_updateState) Task=`8e38d69c-42a0-49ca-8932-ad381cceb23b`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:55:54,587::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:55:54,587::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:55:54,587::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:55:54,587::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:55:54,587::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:55:54,587::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:55:54,588::task::990::TaskManager.Task::(_decref) Task=`8e38d69c-42a0-49ca-8932-ad381cceb23b`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:55:55,898::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:55:55,973::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0334796 s, 30.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:55:56,846::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:55:57,987::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:55:58,071::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0419353 s, 24.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:55:59,214::task::595::TaskManager.Task::(_updateState) Task=`740d617c-b0cb-4326-a831-face9c19409c`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:55:59,221::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:55:59,221::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000790363', 'lastCheck': '6.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:55:59,221::task::1185::TaskManager.Task::(prepare) Task=`740d617c-b0cb-4326-a831-face9c19409c`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000790363', 'lastCheck': '6.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:55:59,222::task::595::TaskManager.Task::(_updateState) Task=`740d617c-b0cb-4326-a831-face9c19409c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:55:59,222::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:55:59,222::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:55:59,222::task::990::TaskManager.Task::(_decref) Task=`740d617c-b0cb-4326-a831-face9c19409c`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:55:59,865::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:56:00,084::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:00,166::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0377967 s, 27.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:56:02,179::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:02,253::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307896 s, 33.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:56:02,356::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:56:02,364::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000684506 s, 6.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:56:02,885::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:56:04,265::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:04,342::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320697 s, 31.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:56:04,609::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:56:04,610::task::595::TaskManager.Task::(_updateState) Task=`803883b1-eba3-4d59-b18b-4e084fdd7222`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:56:04,610::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:56:04,610::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:56:04,610::task::1185::TaskManager.Task::(prepare) Task=`803883b1-eba3-4d59-b18b-4e084fdd7222`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:56:04,610::task::595::TaskManager.Task::(_updateState) Task=`803883b1-eba3-4d59-b18b-4e084fdd7222`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:56:04,611::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:56:04,611::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:56:04,611::task::990::TaskManager.Task::(_decref) Task=`803883b1-eba3-4d59-b18b-4e084fdd7222`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:56:04,615::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:56:04,615::task::595::TaskManager.Task::(_updateState) Task=`f3d075f7-fab2-433e-8047-a6be1e63ad71`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:56:04,616::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:56:04,616::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3d92b2b3-09d4-4430-b1f5-e74a2bf244bd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:56:04,616::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:56:04,617::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:56:04,617::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3d92b2b3-09d4-4430-b1f5-e74a2bf244bd`::Granted request >Thread-338::DEBUG::2014-05-13 11:56:04,617::task::827::TaskManager.Task::(resourceAcquired) Task=`f3d075f7-fab2-433e-8047-a6be1e63ad71`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:56:04,617::task::990::TaskManager.Task::(_decref) Task=`f3d075f7-fab2-433e-8047-a6be1e63ad71`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:56:04,618::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:56:04,618::task::1185::TaskManager.Task::(prepare) Task=`f3d075f7-fab2-433e-8047-a6be1e63ad71`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:56:04,618::task::595::TaskManager.Task::(_updateState) Task=`f3d075f7-fab2-433e-8047-a6be1e63ad71`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:56:04,619::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:56:04,619::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:56:04,619::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:56:04,619::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:56:04,619::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:56:04,620::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:56:04,620::task::990::TaskManager.Task::(_decref) Task=`f3d075f7-fab2-433e-8047-a6be1e63ad71`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:56:05,902::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:56:06,355::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:06,436::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0376281 s, 27.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:56:08,449::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:08,523::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.032722 s, 31.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:56:08,921::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:56:10,536::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:10,611::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0330902 s, 30.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:56:11,942::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:56:12,371::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:56:12,379::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000695224 s, 5.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:56:12,624::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:12,710::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0438443 s, 23.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:56:14,361::task::595::TaskManager.Task::(_updateState) Task=`69c6b117-25a7-4707-ac0a-b4307c43b290`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:56:14,361::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:56:14,361::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000695224', 'lastCheck': '2.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:56:14,361::task::1185::TaskManager.Task::(prepare) Task=`69c6b117-25a7-4707-ac0a-b4307c43b290`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000695224', 'lastCheck': '2.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:56:14,361::task::595::TaskManager.Task::(_updateState) Task=`69c6b117-25a7-4707-ac0a-b4307c43b290`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:56:14,362::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:56:14,362::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:56:14,362::task::990::TaskManager.Task::(_decref) Task=`69c6b117-25a7-4707-ac0a-b4307c43b290`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 11:56:14,638::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:56:14,639::task::595::TaskManager.Task::(_updateState) Task=`13eefd3c-744d-41a1-9d08-2f36288ca984`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:56:14,639::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:56:14,639::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:56:14,639::task::1185::TaskManager.Task::(prepare) Task=`13eefd3c-744d-41a1-9d08-2f36288ca984`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:56:14,640::task::595::TaskManager.Task::(_updateState) Task=`13eefd3c-744d-41a1-9d08-2f36288ca984`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:56:14,640::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:56:14,640::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:56:14,640::task::990::TaskManager.Task::(_decref) Task=`13eefd3c-744d-41a1-9d08-2f36288ca984`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:56:14,644::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:56:14,645::task::595::TaskManager.Task::(_updateState) Task=`564fb661-f50f-4b34-a02f-fd6cb46e19fe`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:56:14,645::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:56:14,645::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`15e4a27d-6125-4538-ab20-9cd5dc81c714`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:56:14,646::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:56:14,646::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:56:14,646::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`15e4a27d-6125-4538-ab20-9cd5dc81c714`::Granted request >Thread-338::DEBUG::2014-05-13 11:56:14,646::task::827::TaskManager.Task::(resourceAcquired) Task=`564fb661-f50f-4b34-a02f-fd6cb46e19fe`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:56:14,647::task::990::TaskManager.Task::(_decref) Task=`564fb661-f50f-4b34-a02f-fd6cb46e19fe`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:56:14,647::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:56:14,647::task::1185::TaskManager.Task::(prepare) Task=`564fb661-f50f-4b34-a02f-fd6cb46e19fe`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:56:14,648::task::595::TaskManager.Task::(_updateState) Task=`564fb661-f50f-4b34-a02f-fd6cb46e19fe`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:56:14,648::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:56:14,648::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:56:14,648::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:56:14,648::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:56:14,649::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:56:14,649::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:56:14,649::task::990::TaskManager.Task::(_decref) Task=`564fb661-f50f-4b34-a02f-fd6cb46e19fe`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:56:14,723::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:14,802::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036588 s, 28.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:56:14,961::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:56:16,815::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:16,890::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0329458 s, 31.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:56:17,981::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:56:18,903::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:18,978::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316532 s, 32.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:56:20,991::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:56:21,003::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:56:21,070::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358412 s, 28.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:56:22,387::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:56:22,395::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000873946 s, 4.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:56:23,083::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:23,169::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0424432 s, 24.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:56:24,020::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:56:24,667::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4a63f6c8] >Thread-13::DEBUG::2014-05-13 11:56:24,668::task::595::TaskManager.Task::(_updateState) Task=`31b31b96-a98d-4f0b-91cd-1b12e744ed77`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:56:24,668::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:56:24,668::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:56:24,668::task::1185::TaskManager.Task::(prepare) Task=`31b31b96-a98d-4f0b-91cd-1b12e744ed77`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:56:24,669::task::595::TaskManager.Task::(_updateState) Task=`31b31b96-a98d-4f0b-91cd-1b12e744ed77`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:56:24,669::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:56:24,669::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:56:24,669::task::990::TaskManager.Task::(_decref) Task=`31b31b96-a98d-4f0b-91cd-1b12e744ed77`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:56:24,673::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4a63f6c8] >Thread-338::DEBUG::2014-05-13 11:56:24,673::task::595::TaskManager.Task::(_updateState) Task=`2f4415ea-615e-4955-91f8-3ce2eaf7e50c`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:56:24,674::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:56:24,674::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`88a200fe-daca-4866-b379-0328e1a1f87c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:56:24,674::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:56:24,674::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:56:24,675::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`88a200fe-daca-4866-b379-0328e1a1f87c`::Granted request >Thread-338::DEBUG::2014-05-13 11:56:24,675::task::827::TaskManager.Task::(resourceAcquired) Task=`2f4415ea-615e-4955-91f8-3ce2eaf7e50c`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:56:24,675::task::990::TaskManager.Task::(_decref) Task=`2f4415ea-615e-4955-91f8-3ce2eaf7e50c`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:56:24,676::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:56:24,676::task::1185::TaskManager.Task::(prepare) Task=`2f4415ea-615e-4955-91f8-3ce2eaf7e50c`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:56:24,676::task::595::TaskManager.Task::(_updateState) Task=`2f4415ea-615e-4955-91f8-3ce2eaf7e50c`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:56:24,676::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:56:24,676::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:56:24,677::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:56:24,677::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:56:24,677::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:56:24,677::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:56:24,677::task::990::TaskManager.Task::(_decref) Task=`2f4415ea-615e-4955-91f8-3ce2eaf7e50c`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:56:25,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:25,267::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0432993 s, 23.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:56:27,039::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:56:27,280::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:27,361::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387884 s, 26.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:56:29,374::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:29,449::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0335216 s, 30.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:56:29,497::task::595::TaskManager.Task::(_updateState) Task=`98bbfb6d-a8e9-44cd-aa72-ed284b9ecc00`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:56:29,498::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:56:29,498::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000873946', 'lastCheck': '7.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:56:29,498::task::1185::TaskManager.Task::(prepare) Task=`98bbfb6d-a8e9-44cd-aa72-ed284b9ecc00`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000873946', 'lastCheck': '7.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:56:29,498::task::595::TaskManager.Task::(_updateState) Task=`98bbfb6d-a8e9-44cd-aa72-ed284b9ecc00`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:56:29,499::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:56:29,499::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:56:29,499::task::990::TaskManager.Task::(_decref) Task=`98bbfb6d-a8e9-44cd-aa72-ed284b9ecc00`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:56:30,060::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:56:31,462::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:31,536::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306639 s, 33.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:56:32,402::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:56:32,410::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000688284 s, 6.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:56:33,078::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:56:33,549::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:33,625::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0332062 s, 30.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:56:34,695::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:56:34,695::task::595::TaskManager.Task::(_updateState) Task=`38bcff85-029a-4941-97d5-5b69a19a66b9`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:56:34,696::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:56:34,696::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:56:34,696::task::1185::TaskManager.Task::(prepare) Task=`38bcff85-029a-4941-97d5-5b69a19a66b9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:56:34,696::task::595::TaskManager.Task::(_updateState) Task=`38bcff85-029a-4941-97d5-5b69a19a66b9`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:56:34,696::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:56:34,697::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:56:34,697::task::990::TaskManager.Task::(_decref) Task=`38bcff85-029a-4941-97d5-5b69a19a66b9`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:56:34,701::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:56:34,701::task::595::TaskManager.Task::(_updateState) Task=`7395391f-022e-47cb-b4f3-71b6ab490cbe`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:56:34,702::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:56:34,702::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e534eb2c-b521-47b6-8577-9f1920d5460e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:56:34,702::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:56:34,703::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:56:34,703::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e534eb2c-b521-47b6-8577-9f1920d5460e`::Granted request >Thread-338::DEBUG::2014-05-13 11:56:34,703::task::827::TaskManager.Task::(resourceAcquired) Task=`7395391f-022e-47cb-b4f3-71b6ab490cbe`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:56:34,703::task::990::TaskManager.Task::(_decref) Task=`7395391f-022e-47cb-b4f3-71b6ab490cbe`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:56:34,704::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:56:34,704::task::1185::TaskManager.Task::(prepare) Task=`7395391f-022e-47cb-b4f3-71b6ab490cbe`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:56:34,704::task::595::TaskManager.Task::(_updateState) Task=`7395391f-022e-47cb-b4f3-71b6ab490cbe`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:56:34,705::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:56:34,705::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:56:34,705::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:56:34,705::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:56:34,705::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:56:34,706::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:56:34,706::task::990::TaskManager.Task::(_decref) Task=`7395391f-022e-47cb-b4f3-71b6ab490cbe`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:56:35,638::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:35,718::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0377055 s, 27.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:56:36,096::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:56:37,731::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:37,805::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311744 s, 32.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:56:39,119::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:56:39,818::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:39,907::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0467075 s, 21.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:56:41,920::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:41,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355626 s, 28.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:56:42,142::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:56:42,417::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:56:42,426::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00125015 s, 3.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:56:44,012::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:44,085::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306505 s, 33.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:56:44,633::task::595::TaskManager.Task::(_updateState) Task=`56f1a815-bd0e-4804-9ad5-a0ee9609d1db`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:56:44,633::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:56:44,634::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00125015', 'lastCheck': '2.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:56:44,634::task::1185::TaskManager.Task::(prepare) Task=`56f1a815-bd0e-4804-9ad5-a0ee9609d1db`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00125015', 'lastCheck': '2.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:56:44,634::task::595::TaskManager.Task::(_updateState) Task=`56f1a815-bd0e-4804-9ad5-a0ee9609d1db`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:56:44,634::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:56:44,634::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:56:44,635::task::990::TaskManager.Task::(_decref) Task=`56f1a815-bd0e-4804-9ad5-a0ee9609d1db`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 11:56:44,724::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:56:44,725::task::595::TaskManager.Task::(_updateState) Task=`8e1f5db4-7ec2-47ec-8fbd-af8302c6a200`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:56:44,725::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:56:44,725::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:56:44,725::task::1185::TaskManager.Task::(prepare) Task=`8e1f5db4-7ec2-47ec-8fbd-af8302c6a200`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:56:44,726::task::595::TaskManager.Task::(_updateState) Task=`8e1f5db4-7ec2-47ec-8fbd-af8302c6a200`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:56:44,726::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:56:44,726::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:56:44,726::task::990::TaskManager.Task::(_decref) Task=`8e1f5db4-7ec2-47ec-8fbd-af8302c6a200`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:56:44,770::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:56:44,771::task::595::TaskManager.Task::(_updateState) Task=`3e14fbf6-9a01-4064-a70d-2ef0c32eb32f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:56:44,771::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:56:44,771::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`154e446f-2b3e-47b4-940b-d65c64491ba0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:56:44,772::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:56:44,772::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:56:44,772::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`154e446f-2b3e-47b4-940b-d65c64491ba0`::Granted request >Thread-338::DEBUG::2014-05-13 11:56:44,773::task::827::TaskManager.Task::(resourceAcquired) Task=`3e14fbf6-9a01-4064-a70d-2ef0c32eb32f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:56:44,773::task::990::TaskManager.Task::(_decref) Task=`3e14fbf6-9a01-4064-a70d-2ef0c32eb32f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:56:44,774::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:56:44,774::task::1185::TaskManager.Task::(prepare) Task=`3e14fbf6-9a01-4064-a70d-2ef0c32eb32f`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:56:44,775::task::595::TaskManager.Task::(_updateState) Task=`3e14fbf6-9a01-4064-a70d-2ef0c32eb32f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:56:44,775::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:56:44,775::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:56:44,775::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:56:44,775::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:56:44,776::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:56:44,776::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:56:44,776::task::990::TaskManager.Task::(_decref) Task=`3e14fbf6-9a01-4064-a70d-2ef0c32eb32f`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:56:45,162::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:56:46,098::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:46,173::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0330033 s, 31.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:56:48,181::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:56:48,186::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:48,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.038112 s, 26.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:56:50,280::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:50,359::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381293 s, 26.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:56:51,200::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:56:52,372::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 11:56:52,453::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:56:52,470::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000746639 s, 5.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:56:52,508::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0337821 s, 30.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:56:54,232::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:56:54,521::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:54,611::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0470532 s, 21.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:56:54,794::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:56:54,795::task::595::TaskManager.Task::(_updateState) Task=`44948100-4fd7-4288-9bdf-3fe657bb2351`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:56:54,795::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:56:54,795::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:56:54,796::task::1185::TaskManager.Task::(prepare) Task=`44948100-4fd7-4288-9bdf-3fe657bb2351`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:56:54,796::task::595::TaskManager.Task::(_updateState) Task=`44948100-4fd7-4288-9bdf-3fe657bb2351`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:56:54,796::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:56:54,796::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:56:54,796::task::990::TaskManager.Task::(_decref) Task=`44948100-4fd7-4288-9bdf-3fe657bb2351`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:56:54,801::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:56:54,801::task::595::TaskManager.Task::(_updateState) Task=`b3b49da6-3d29-4f2e-b5a5-3a28ac80138e`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:56:54,801::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:56:54,802::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5cbafd44-a199-40e2-831a-a12e4bb2048b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:56:54,802::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:56:54,802::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:56:54,802::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5cbafd44-a199-40e2-831a-a12e4bb2048b`::Granted request >Thread-338::DEBUG::2014-05-13 11:56:54,803::task::827::TaskManager.Task::(resourceAcquired) Task=`b3b49da6-3d29-4f2e-b5a5-3a28ac80138e`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:56:54,803::task::990::TaskManager.Task::(_decref) Task=`b3b49da6-3d29-4f2e-b5a5-3a28ac80138e`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:56:54,804::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:56:54,804::task::1185::TaskManager.Task::(prepare) Task=`b3b49da6-3d29-4f2e-b5a5-3a28ac80138e`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:56:54,804::task::595::TaskManager.Task::(_updateState) Task=`b3b49da6-3d29-4f2e-b5a5-3a28ac80138e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:56:54,804::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:56:54,804::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:56:54,805::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:56:54,805::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:56:54,805::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:56:54,805::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:56:54,805::task::990::TaskManager.Task::(_decref) Task=`b3b49da6-3d29-4f2e-b5a5-3a28ac80138e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:56:56,628::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:56,707::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374544 s, 27.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:56:57,254::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:56:58,720::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:56:58,796::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0336772 s, 30.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:56:59,766::task::595::TaskManager.Task::(_updateState) Task=`481b0b83-c760-4c30-b584-190c083d1969`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:56:59,767::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:56:59,767::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000746639', 'lastCheck': '7.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:56:59,767::task::1185::TaskManager.Task::(prepare) Task=`481b0b83-c760-4c30-b584-190c083d1969`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000746639', 'lastCheck': '7.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:56:59,767::task::595::TaskManager.Task::(_updateState) Task=`481b0b83-c760-4c30-b584-190c083d1969`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:56:59,768::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:56:59,768::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:56:59,768::task::990::TaskManager.Task::(_decref) Task=`481b0b83-c760-4c30-b584-190c083d1969`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:57:00,273::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:57:00,809::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:00,883::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0308443 s, 33.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:57:02,497::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:57:02,505::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00078123 s, 5.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:57:02,896::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:02,977::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0385536 s, 26.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:57:03,299::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:57:04,825::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 11:57:04,825::task::595::TaskManager.Task::(_updateState) Task=`2e40a9c2-e4ed-424d-87ef-15a248c552ae`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:57:04,826::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:57:04,826::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:57:04,826::task::1185::TaskManager.Task::(prepare) Task=`2e40a9c2-e4ed-424d-87ef-15a248c552ae`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:57:04,826::task::595::TaskManager.Task::(_updateState) Task=`2e40a9c2-e4ed-424d-87ef-15a248c552ae`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:57:04,827::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:57:04,827::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:57:04,827::task::990::TaskManager.Task::(_decref) Task=`2e40a9c2-e4ed-424d-87ef-15a248c552ae`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:57:04,832::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:57:04,832::task::595::TaskManager.Task::(_updateState) Task=`38019449-032d-45ca-aed4-298fd011a926`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:57:04,832::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:57:04,833::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2d9b04d8-e478-4087-b0bf-4fdae0d91679`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:57:04,833::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:57:04,833::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:57:04,833::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2d9b04d8-e478-4087-b0bf-4fdae0d91679`::Granted request >Thread-338::DEBUG::2014-05-13 11:57:04,834::task::827::TaskManager.Task::(resourceAcquired) Task=`38019449-032d-45ca-aed4-298fd011a926`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:57:04,834::task::990::TaskManager.Task::(_decref) Task=`38019449-032d-45ca-aed4-298fd011a926`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:57:04,835::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:57:04,835::task::1185::TaskManager.Task::(prepare) Task=`38019449-032d-45ca-aed4-298fd011a926`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:57:04,835::task::595::TaskManager.Task::(_updateState) Task=`38019449-032d-45ca-aed4-298fd011a926`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:57:04,835::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:57:04,835::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:57:04,836::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:57:04,836::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:57:04,836::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:57:04,836::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:57:04,836::task::990::TaskManager.Task::(_decref) Task=`38019449-032d-45ca-aed4-298fd011a926`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:57:04,990::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:05,073::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383797 s, 26.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:57:06,318::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:57:07,086::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:07,192::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0637549 s, 16.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:57:09,205::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:09,284::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363022 s, 28.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:57:09,336::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:57:11,297::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:11,378::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0389149 s, 26.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:57:12,354::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:57:12,512::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:57:12,520::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000698819 s, 5.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:57:13,391::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:13,466::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0331137 s, 30.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:57:14,857::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4478bd69] >Thread-13::DEBUG::2014-05-13 11:57:14,858::task::595::TaskManager.Task::(_updateState) Task=`74976efb-d1e1-4272-adcb-3c700f7a2418`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:57:14,858::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:57:14,858::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:57:14,858::task::1185::TaskManager.Task::(prepare) Task=`74976efb-d1e1-4272-adcb-3c700f7a2418`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:57:14,859::task::595::TaskManager.Task::(_updateState) Task=`74976efb-d1e1-4272-adcb-3c700f7a2418`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:57:14,859::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:57:14,859::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:57:14,859::task::990::TaskManager.Task::(_decref) Task=`74976efb-d1e1-4272-adcb-3c700f7a2418`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:57:14,864::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4478bd69] >Thread-338::DEBUG::2014-05-13 11:57:14,864::task::595::TaskManager.Task::(_updateState) Task=`cbd7677e-623d-4439-9c58-0fd35625fed8`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:57:14,864::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:57:14,865::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f51c3974-3b0c-477c-a891-570fd66d7199`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:57:14,865::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:57:14,865::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:57:14,865::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f51c3974-3b0c-477c-a891-570fd66d7199`::Granted request >Thread-338::DEBUG::2014-05-13 11:57:14,866::task::827::TaskManager.Task::(resourceAcquired) Task=`cbd7677e-623d-4439-9c58-0fd35625fed8`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:57:14,866::task::990::TaskManager.Task::(_decref) Task=`cbd7677e-623d-4439-9c58-0fd35625fed8`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:57:14,866::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:57:14,867::task::1185::TaskManager.Task::(prepare) Task=`cbd7677e-623d-4439-9c58-0fd35625fed8`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:57:14,867::task::595::TaskManager.Task::(_updateState) Task=`cbd7677e-623d-4439-9c58-0fd35625fed8`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:57:14,867::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:57:14,867::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:57:14,867::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:57:14,868::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:57:14,868::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:57:14,868::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:57:14,868::task::990::TaskManager.Task::(_decref) Task=`cbd7677e-623d-4439-9c58-0fd35625fed8`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 11:57:14,898::task::595::TaskManager.Task::(_updateState) Task=`7c5637ad-edd7-46bd-9e40-ac14309d34ba`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:57:14,898::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:57:14,898::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000698819', 'lastCheck': '2.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:57:14,898::task::1185::TaskManager.Task::(prepare) Task=`7c5637ad-edd7-46bd-9e40-ac14309d34ba`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000698819', 'lastCheck': '2.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:57:14,899::task::595::TaskManager.Task::(_updateState) Task=`7c5637ad-edd7-46bd-9e40-ac14309d34ba`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:57:14,899::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:57:14,899::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:57:14,899::task::990::TaskManager.Task::(_decref) Task=`7c5637ad-edd7-46bd-9e40-ac14309d34ba`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:57:15,373::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:57:15,479::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:15,555::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360516 s, 28.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:57:17,565::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:17,640::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315076 s, 32.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:57:18,392::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:57:19,653::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:19,735::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323473 s, 31.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:57:21,411::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:57:21,750::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:21,836::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0421824 s, 24.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:57:22,528::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:57:22,536::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000687637 s, 6.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:57:23,849::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:23,931::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037249 s, 27.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:57:24,429::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 11:57:24,886::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17998964] >Thread-13::DEBUG::2014-05-13 11:57:24,886::task::595::TaskManager.Task::(_updateState) Task=`bd33149f-3c57-4537-b985-9c25b3b2e9cd`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:57:24,886::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:57:24,887::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:57:24,887::task::1185::TaskManager.Task::(prepare) Task=`bd33149f-3c57-4537-b985-9c25b3b2e9cd`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:57:24,887::task::595::TaskManager.Task::(_updateState) Task=`bd33149f-3c57-4537-b985-9c25b3b2e9cd`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:57:24,887::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:57:24,888::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:57:24,888::task::990::TaskManager.Task::(_decref) Task=`bd33149f-3c57-4537-b985-9c25b3b2e9cd`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:57:24,892::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17998964] >Thread-338::DEBUG::2014-05-13 11:57:24,893::task::595::TaskManager.Task::(_updateState) Task=`ca53e48b-faf3-4080-bd52-dadb7c2c08f7`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:57:24,893::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:57:24,893::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6e2e4602-eebe-47b5-ab1c-dae54d701c29`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:57:24,893::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:57:24,894::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:57:24,894::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6e2e4602-eebe-47b5-ab1c-dae54d701c29`::Granted request >Thread-338::DEBUG::2014-05-13 11:57:24,894::task::827::TaskManager.Task::(resourceAcquired) Task=`ca53e48b-faf3-4080-bd52-dadb7c2c08f7`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:57:24,894::task::990::TaskManager.Task::(_decref) Task=`ca53e48b-faf3-4080-bd52-dadb7c2c08f7`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:57:24,895::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:57:24,896::task::1185::TaskManager.Task::(prepare) Task=`ca53e48b-faf3-4080-bd52-dadb7c2c08f7`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:57:24,896::task::595::TaskManager.Task::(_updateState) Task=`ca53e48b-faf3-4080-bd52-dadb7c2c08f7`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:57:24,896::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:57:24,896::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:57:24,896::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:57:24,897::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:57:24,897::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:57:24,897::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:57:24,897::task::990::TaskManager.Task::(_decref) Task=`ca53e48b-faf3-4080-bd52-dadb7c2c08f7`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:57:25,944::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:26,026::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383443 s, 26.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:57:27,449::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:57:28,039::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:28,114::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317723 s, 32.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:57:30,031::task::595::TaskManager.Task::(_updateState) Task=`b35ab58e-f4d1-4d71-b7b8-6eefdf9ab471`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:57:30,032::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 11:57:30,032::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000687637', 'lastCheck': '7.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:57:30,032::task::1185::TaskManager.Task::(prepare) Task=`b35ab58e-f4d1-4d71-b7b8-6eefdf9ab471`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000687637', 'lastCheck': '7.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 11:57:30,032::task::595::TaskManager.Task::(_updateState) Task=`b35ab58e-f4d1-4d71-b7b8-6eefdf9ab471`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:57:30,032::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:57:30,033::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:57:30,033::task::990::TaskManager.Task::(_decref) Task=`b35ab58e-f4d1-4d71-b7b8-6eefdf9ab471`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:57:30,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:30,209::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0392591 s, 26.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:57:30,469::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:57:32,222::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:32,297::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307341 s, 33.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:57:32,543::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:57:32,551::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000685678 s, 6.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:57:33,487::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:57:34,310::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:34,761::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.409377 s, 2.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:57:34,916::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [28fd42b9] >Thread-13::DEBUG::2014-05-13 11:57:34,916::task::595::TaskManager.Task::(_updateState) Task=`9b3f2bec-eba3-45f7-8e15-73ad6879040d`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:57:34,916::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:57:34,917::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:57:34,917::task::1185::TaskManager.Task::(prepare) Task=`9b3f2bec-eba3-45f7-8e15-73ad6879040d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:57:34,917::task::595::TaskManager.Task::(_updateState) Task=`9b3f2bec-eba3-45f7-8e15-73ad6879040d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:57:34,917::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:57:34,917::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:57:34,918::task::990::TaskManager.Task::(_decref) Task=`9b3f2bec-eba3-45f7-8e15-73ad6879040d`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:57:34,922::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [28fd42b9] >Thread-338::DEBUG::2014-05-13 11:57:34,922::task::595::TaskManager.Task::(_updateState) Task=`b9d4a848-8f78-4f21-9a58-689f81256385`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:57:34,922::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:57:34,923::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`beaa9fc8-7562-46cd-a510-6d396f6a5fc2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:57:34,923::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:57:34,923::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:57:34,923::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`beaa9fc8-7562-46cd-a510-6d396f6a5fc2`::Granted request >Thread-338::DEBUG::2014-05-13 11:57:34,924::task::827::TaskManager.Task::(resourceAcquired) Task=`b9d4a848-8f78-4f21-9a58-689f81256385`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:57:34,924::task::990::TaskManager.Task::(_decref) Task=`b9d4a848-8f78-4f21-9a58-689f81256385`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:57:34,925::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:57:34,925::task::1185::TaskManager.Task::(prepare) Task=`b9d4a848-8f78-4f21-9a58-689f81256385`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:57:34,925::task::595::TaskManager.Task::(_updateState) Task=`b9d4a848-8f78-4f21-9a58-689f81256385`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:57:34,925::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:57:34,925::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:57:34,926::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:57:34,926::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:57:34,926::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:57:34,926::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:57:34,926::task::990::TaskManager.Task::(_decref) Task=`b9d4a848-8f78-4f21-9a58-689f81256385`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:57:36,512::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:57:36,774::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:36,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318371 s, 32.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:57:38,864::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:38,943::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0350291 s, 29.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:57:39,529::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:57:40,956::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:41,034::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357124 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:57:42,546::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:57:42,558::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:57:42,566::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000638058 s, 6.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:57:43,047::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:43,129::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379173 s, 27.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 11:57:44,946::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5eff0b16] >Thread-13::DEBUG::2014-05-13 11:57:44,947::task::595::TaskManager.Task::(_updateState) Task=`30473e4c-786a-4ca8-bef2-5b9c7a02c8e6`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 11:57:44,947::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 11:57:44,947::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:57:44,948::task::1185::TaskManager.Task::(prepare) Task=`30473e4c-786a-4ca8-bef2-5b9c7a02c8e6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 11:57:44,948::task::595::TaskManager.Task::(_updateState) Task=`30473e4c-786a-4ca8-bef2-5b9c7a02c8e6`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 11:57:44,948::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 11:57:44,948::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 11:57:44,948::task::990::TaskManager.Task::(_decref) Task=`30473e4c-786a-4ca8-bef2-5b9c7a02c8e6`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:57:45,142::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 11:57:45,175::task::595::TaskManager.Task::(_updateState) Task=`ee164585-1459-41a9-a273-b087b3119309`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:57:45,176::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:57:45,178::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000638058', 'lastCheck': '2.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:57:45,178::task::1185::TaskManager.Task::(prepare) Task=`ee164585-1459-41a9-a273-b087b3119309`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000638058', 'lastCheck': '2.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:57:45,179::task::595::TaskManager.Task::(_updateState) Task=`ee164585-1459-41a9-a273-b087b3119309`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:57:45,180::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:57:45,181::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:57:45,182::task::990::TaskManager.Task::(_decref) Task=`ee164585-1459-41a9-a273-b087b3119309`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:57:45,224::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5eff0b16] >Thread-338::DEBUG::2014-05-13 11:57:45,226::task::595::TaskManager.Task::(_updateState) Task=`15a954f9-84d4-46b1-baad-fd9d6800b842`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:57:45,227::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:57:45,229::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ba2ac76b-9068-42fe-ab50-2dad81189056`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:57:45,230::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:57:45,231::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:57:45,232::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ba2ac76b-9068-42fe-ab50-2dad81189056`::Granted request >Thread-338::DEBUG::2014-05-13 11:57:45,234::task::827::TaskManager.Task::(resourceAcquired) Task=`15a954f9-84d4-46b1-baad-fd9d6800b842`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:57:45,235::task::990::TaskManager.Task::(_decref) Task=`15a954f9-84d4-46b1-baad-fd9d6800b842`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:57:45,242::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:57:45,243::task::1185::TaskManager.Task::(prepare) Task=`15a954f9-84d4-46b1-baad-fd9d6800b842`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:57:45,243::task::595::TaskManager.Task::(_updateState) Task=`15a954f9-84d4-46b1-baad-fd9d6800b842`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:57:45,244::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:57:45,245::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:57:45,246::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:57:45,248::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:57:45,248::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:57:45,249::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:57:45,251::task::990::TaskManager.Task::(_decref) Task=`15a954f9-84d4-46b1-baad-fd9d6800b842`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:57:45,276::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.058747 s, 17.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:57:45,564::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:57:47,288::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:47,373::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0429327 s, 23.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:57:48,585::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:57:49,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:49,461::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0333422 s, 30.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:57:51,474::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:51,555::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0378109 s, 27.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:57:51,602::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:57:52,573::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:57:52,581::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000683188 s, 6.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:57:53,568::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:53,648::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359143 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:57:54,625::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:57:55,273::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:57:55,274::task::595::TaskManager.Task::(_updateState) Task=`d51dbbbe-b86e-43a3-b40c-5aa47a5f14c2`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:57:55,274::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:57:55,274::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:57:55,275::task::1185::TaskManager.Task::(prepare) Task=`d51dbbbe-b86e-43a3-b40c-5aa47a5f14c2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:57:55,275::task::595::TaskManager.Task::(_updateState) Task=`d51dbbbe-b86e-43a3-b40c-5aa47a5f14c2`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:57:55,275::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:57:55,275::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:57:55,275::task::990::TaskManager.Task::(_decref) Task=`d51dbbbe-b86e-43a3-b40c-5aa47a5f14c2`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:57:55,284::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:57:55,284::task::595::TaskManager.Task::(_updateState) Task=`4cfa536d-3264-4fae-b8f3-bb78c96c1b85`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:57:55,284::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:57:55,285::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5fcbe593-d9e7-48df-ab96-4743b7e5604a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:57:55,285::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:57:55,285::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:57:55,285::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5fcbe593-d9e7-48df-ab96-4743b7e5604a`::Granted request >Thread-338::DEBUG::2014-05-13 11:57:55,287::task::827::TaskManager.Task::(resourceAcquired) Task=`4cfa536d-3264-4fae-b8f3-bb78c96c1b85`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:57:55,287::task::990::TaskManager.Task::(_decref) Task=`4cfa536d-3264-4fae-b8f3-bb78c96c1b85`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:57:55,288::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:57:55,288::task::1185::TaskManager.Task::(prepare) Task=`4cfa536d-3264-4fae-b8f3-bb78c96c1b85`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:57:55,288::task::595::TaskManager.Task::(_updateState) Task=`4cfa536d-3264-4fae-b8f3-bb78c96c1b85`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:57:55,288::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:57:55,288::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:57:55,289::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:57:55,289::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:57:55,289::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:57:55,289::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:57:55,289::task::990::TaskManager.Task::(_decref) Task=`4cfa536d-3264-4fae-b8f3-bb78c96c1b85`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:57:55,661::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:55,740::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360991 s, 28.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:57:57,645::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:57:57,753::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:57,826::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311013 s, 32.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:57:59,840::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:57:59,931::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0388296 s, 26.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:58:00,356::task::595::TaskManager.Task::(_updateState) Task=`f48d6bbf-af56-4f2a-8907-76c99444c7e2`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:58:00,356::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:58:00,357::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000683188', 'lastCheck': '7.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:58:00,357::task::1185::TaskManager.Task::(prepare) Task=`f48d6bbf-af56-4f2a-8907-76c99444c7e2`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000683188', 'lastCheck': '7.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:58:00,357::task::595::TaskManager.Task::(_updateState) Task=`f48d6bbf-af56-4f2a-8907-76c99444c7e2`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:58:00,357::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:58:00,357::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:58:00,357::task::990::TaskManager.Task::(_decref) Task=`f48d6bbf-af56-4f2a-8907-76c99444c7e2`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:58:00,664::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:58:01,944::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:58:02,300::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.312386 s, 3.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:58:02,589::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:58:02,597::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000727441 s, 5.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:58:03,683::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:58:04,313::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:58:04,388::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306197 s, 33.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:58:05,308::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:58:05,309::task::595::TaskManager.Task::(_updateState) Task=`3ad04a2f-b607-45ef-9103-d4d9b5239be5`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:58:05,309::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:58:05,309::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:58:05,309::task::1185::TaskManager.Task::(prepare) Task=`3ad04a2f-b607-45ef-9103-d4d9b5239be5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:58:05,310::task::595::TaskManager.Task::(_updateState) Task=`3ad04a2f-b607-45ef-9103-d4d9b5239be5`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:58:05,310::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:58:05,310::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:58:05,310::task::990::TaskManager.Task::(_decref) Task=`3ad04a2f-b607-45ef-9103-d4d9b5239be5`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:58:05,314::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:58:05,315::task::595::TaskManager.Task::(_updateState) Task=`b139bb45-99da-445c-b154-e02da0401429`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:58:05,315::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:58:05,315::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`73105078-6226-49a8-a317-41f448a9e3c4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:58:05,316::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:58:05,316::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:58:05,316::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`73105078-6226-49a8-a317-41f448a9e3c4`::Granted request >Thread-338::DEBUG::2014-05-13 11:58:05,316::task::827::TaskManager.Task::(resourceAcquired) Task=`b139bb45-99da-445c-b154-e02da0401429`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:58:05,317::task::990::TaskManager.Task::(_decref) Task=`b139bb45-99da-445c-b154-e02da0401429`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:58:05,317::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:58:05,318::task::1185::TaskManager.Task::(prepare) Task=`b139bb45-99da-445c-b154-e02da0401429`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:58:05,318::task::595::TaskManager.Task::(_updateState) Task=`b139bb45-99da-445c-b154-e02da0401429`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:58:05,318::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:58:05,318::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:58:05,318::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:58:05,319::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:58:05,319::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:58:05,319::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:58:05,319::task::990::TaskManager.Task::(_decref) Task=`b139bb45-99da-445c-b154-e02da0401429`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:58:06,401::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:58:06,487::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381779 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:58:06,704::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:58:08,500::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:58:08,576::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318956 s, 32.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:58:09,729::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:58:10,589::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:58:10,668::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373559 s, 27.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:58:12,604::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:58:12,611::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00063289 s, 6.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:58:12,681::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:58:12,749::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:58:13,043::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.319081 s, 3.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:58:15,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:58:15,135::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353673 s, 29.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:58:15,339::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:58:15,339::task::595::TaskManager.Task::(_updateState) Task=`bbbf1173-ae23-499f-910d-cc8280437ecb`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:58:15,339::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:58:15,340::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:58:15,340::task::1185::TaskManager.Task::(prepare) Task=`bbbf1173-ae23-499f-910d-cc8280437ecb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:58:15,340::task::595::TaskManager.Task::(_updateState) Task=`bbbf1173-ae23-499f-910d-cc8280437ecb`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:58:15,340::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:58:15,340::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:58:15,341::task::990::TaskManager.Task::(_decref) Task=`bbbf1173-ae23-499f-910d-cc8280437ecb`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:58:15,345::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:58:15,345::task::595::TaskManager.Task::(_updateState) Task=`233647dc-116a-472f-a066-4d0f0838e22e`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:58:15,345::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:58:15,346::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9fce8ccf-3928-4daf-8008-a68351b3e5c6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:58:15,346::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:58:15,346::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:58:15,346::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9fce8ccf-3928-4daf-8008-a68351b3e5c6`::Granted request >Thread-338::DEBUG::2014-05-13 11:58:15,347::task::827::TaskManager.Task::(resourceAcquired) Task=`233647dc-116a-472f-a066-4d0f0838e22e`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:58:15,347::task::990::TaskManager.Task::(_decref) Task=`233647dc-116a-472f-a066-4d0f0838e22e`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:58:15,348::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:58:15,348::task::1185::TaskManager.Task::(prepare) Task=`233647dc-116a-472f-a066-4d0f0838e22e`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:58:15,348::task::595::TaskManager.Task::(_updateState) Task=`233647dc-116a-472f-a066-4d0f0838e22e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:58:15,348::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:58:15,348::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:58:15,349::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:58:15,349::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:58:15,349::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:58:15,349::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:58:15,349::task::990::TaskManager.Task::(_decref) Task=`233647dc-116a-472f-a066-4d0f0838e22e`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 11:58:15,491::task::595::TaskManager.Task::(_updateState) Task=`350afd11-bea7-43d6-8320-d4ad20426cb5`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:58:15,491::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:58:15,491::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00063289', 'lastCheck': '2.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:58:15,491::task::1185::TaskManager.Task::(prepare) Task=`350afd11-bea7-43d6-8320-d4ad20426cb5`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00063289', 'lastCheck': '2.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:58:15,492::task::595::TaskManager.Task::(_updateState) Task=`350afd11-bea7-43d6-8320-d4ad20426cb5`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:58:15,492::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:58:15,492::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:58:15,492::task::990::TaskManager.Task::(_decref) Task=`350afd11-bea7-43d6-8320-d4ad20426cb5`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:58:15,768::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:58:17,148::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:58:17,224::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313811 s, 32.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:58:18,787::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:58:19,237::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:58:19,311::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310583 s, 33.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:58:21,324::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:58:21,416::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0354795 s, 28.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:58:21,806::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:58:22,618::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:58:22,626::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000658586 s, 6.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:58:23,429::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:58:23,515::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.041511 s, 24.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:58:24,825::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:58:25,368::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [7a000b17] >Thread-66::DEBUG::2014-05-13 11:58:25,369::task::595::TaskManager.Task::(_updateState) Task=`ad5f670c-61e4-448f-8ae9-88f3f5dc8c06`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:58:25,369::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:58:25,369::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:58:25,370::task::1185::TaskManager.Task::(prepare) Task=`ad5f670c-61e4-448f-8ae9-88f3f5dc8c06`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:58:25,370::task::595::TaskManager.Task::(_updateState) Task=`ad5f670c-61e4-448f-8ae9-88f3f5dc8c06`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:58:25,370::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:58:25,370::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:58:25,370::task::990::TaskManager.Task::(_decref) Task=`ad5f670c-61e4-448f-8ae9-88f3f5dc8c06`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:58:25,376::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [7a000b17] >Thread-338::DEBUG::2014-05-13 11:58:25,376::task::595::TaskManager.Task::(_updateState) Task=`344b8397-b65d-4f4d-a101-6e566b32b9f5`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:58:25,376::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:58:25,377::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c65a6997-6264-46c7-813b-87c6f7e872e8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:58:25,377::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:58:25,377::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:58:25,378::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c65a6997-6264-46c7-813b-87c6f7e872e8`::Granted request >Thread-338::DEBUG::2014-05-13 11:58:25,378::task::827::TaskManager.Task::(resourceAcquired) Task=`344b8397-b65d-4f4d-a101-6e566b32b9f5`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:58:25,378::task::990::TaskManager.Task::(_decref) Task=`344b8397-b65d-4f4d-a101-6e566b32b9f5`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:58:25,379::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:58:25,379::task::1185::TaskManager.Task::(prepare) Task=`344b8397-b65d-4f4d-a101-6e566b32b9f5`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:58:25,379::task::595::TaskManager.Task::(_updateState) Task=`344b8397-b65d-4f4d-a101-6e566b32b9f5`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:58:25,379::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:58:25,380::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:58:25,380::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:58:25,380::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:58:25,380::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:58:25,380::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:58:25,381::task::990::TaskManager.Task::(_decref) Task=`344b8397-b65d-4f4d-a101-6e566b32b9f5`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:58:25,528::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:58:25,905::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.335011 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:58:27,847::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:58:27,918::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:58:27,995::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0330096 s, 31.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:58:30,008::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:58:30,092::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0394719 s, 25.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:58:30,622::task::595::TaskManager.Task::(_updateState) Task=`48fd6778-cf4f-4641-bc7b-2bdba1558cc7`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:58:30,623::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:58:30,623::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000658586', 'lastCheck': '8.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:58:30,623::task::1185::TaskManager.Task::(prepare) Task=`48fd6778-cf4f-4641-bc7b-2bdba1558cc7`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000658586', 'lastCheck': '8.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:58:30,623::task::595::TaskManager.Task::(_updateState) Task=`48fd6778-cf4f-4641-bc7b-2bdba1558cc7`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:58:30,623::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:58:30,623::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:58:30,624::task::990::TaskManager.Task::(_decref) Task=`48fd6778-cf4f-4641-bc7b-2bdba1558cc7`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:58:30,865::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:58:32,105::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:58:32,179::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320354 s, 32.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:58:32,634::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:58:32,642::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000682457 s, 6.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:58:33,883::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:58:34,192::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:58:34,269::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319183 s, 32.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:58:35,400::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:58:35,401::task::595::TaskManager.Task::(_updateState) Task=`6bdcde55-004e-4c31-ab27-5c583f3bc016`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:58:35,401::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:58:35,402::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:58:35,402::task::1185::TaskManager.Task::(prepare) Task=`6bdcde55-004e-4c31-ab27-5c583f3bc016`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:58:35,402::task::595::TaskManager.Task::(_updateState) Task=`6bdcde55-004e-4c31-ab27-5c583f3bc016`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:58:35,402::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:58:35,402::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:58:35,403::task::990::TaskManager.Task::(_decref) Task=`6bdcde55-004e-4c31-ab27-5c583f3bc016`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:58:35,408::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:58:35,408::task::595::TaskManager.Task::(_updateState) Task=`0e813b54-53a8-4a90-b578-d69dd46680ff`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:58:35,408::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:58:35,409::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`31d6ed35-6932-4f68-80dd-7214d8fd1073`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:58:35,409::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:58:35,409::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:58:35,410::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`31d6ed35-6932-4f68-80dd-7214d8fd1073`::Granted request >Thread-338::DEBUG::2014-05-13 11:58:35,410::task::827::TaskManager.Task::(resourceAcquired) Task=`0e813b54-53a8-4a90-b578-d69dd46680ff`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:58:35,410::task::990::TaskManager.Task::(_decref) Task=`0e813b54-53a8-4a90-b578-d69dd46680ff`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:58:35,411::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:58:35,411::task::1185::TaskManager.Task::(prepare) Task=`0e813b54-53a8-4a90-b578-d69dd46680ff`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:58:35,411::task::595::TaskManager.Task::(_updateState) Task=`0e813b54-53a8-4a90-b578-d69dd46680ff`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:58:35,412::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:58:35,412::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:58:35,412::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:58:35,412::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:58:35,412::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:58:35,412::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:58:35,413::task::990::TaskManager.Task::(_decref) Task=`0e813b54-53a8-4a90-b578-d69dd46680ff`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:58:36,282::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:58:36,363::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363319 s, 28.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:58:36,902::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:58:38,376::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:58:38,465::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0455671 s, 22.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:58:39,921::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:58:40,478::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:58:40,569::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0477752 s, 21.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:58:42,582::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 11:58:42,677::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:58:42,695::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000555473 s, 7.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:58:42,716::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353594 s, 29.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:58:42,938::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:58:44,729::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:58:44,810::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0371804 s, 27.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:58:45,432::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:58:45,432::task::595::TaskManager.Task::(_updateState) Task=`8b27af06-8b5e-4cd1-ae34-167690e377f2`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:58:45,433::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:58:45,433::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:58:45,433::task::1185::TaskManager.Task::(prepare) Task=`8b27af06-8b5e-4cd1-ae34-167690e377f2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:58:45,433::task::595::TaskManager.Task::(_updateState) Task=`8b27af06-8b5e-4cd1-ae34-167690e377f2`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:58:45,434::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:58:45,434::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:58:45,434::task::990::TaskManager.Task::(_decref) Task=`8b27af06-8b5e-4cd1-ae34-167690e377f2`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:58:45,439::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:58:45,439::task::595::TaskManager.Task::(_updateState) Task=`82f49dea-0385-41c5-92e7-95a522b02798`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:58:45,439::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:58:45,440::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0ebba866-4011-4be2-bef5-de95c3dec150`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:58:45,440::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:58:45,440::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:58:45,440::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0ebba866-4011-4be2-bef5-de95c3dec150`::Granted request >Thread-338::DEBUG::2014-05-13 11:58:45,441::task::827::TaskManager.Task::(resourceAcquired) Task=`82f49dea-0385-41c5-92e7-95a522b02798`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:58:45,441::task::990::TaskManager.Task::(_decref) Task=`82f49dea-0385-41c5-92e7-95a522b02798`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:58:45,442::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:58:45,442::task::1185::TaskManager.Task::(prepare) Task=`82f49dea-0385-41c5-92e7-95a522b02798`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:58:45,442::task::595::TaskManager.Task::(_updateState) Task=`82f49dea-0385-41c5-92e7-95a522b02798`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:58:45,442::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:58:45,442::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:58:45,443::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:58:45,443::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:58:45,443::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:58:45,443::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:58:45,443::task::990::TaskManager.Task::(_decref) Task=`82f49dea-0385-41c5-92e7-95a522b02798`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 11:58:45,753::task::595::TaskManager.Task::(_updateState) Task=`717547e7-8f58-4366-ad9a-786555254793`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:58:45,754::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:58:45,754::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000555473', 'lastCheck': '3.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:58:45,754::task::1185::TaskManager.Task::(prepare) Task=`717547e7-8f58-4366-ad9a-786555254793`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000555473', 'lastCheck': '3.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:58:45,754::task::595::TaskManager.Task::(_updateState) Task=`717547e7-8f58-4366-ad9a-786555254793`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:58:45,754::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:58:45,754::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:58:45,755::task::990::TaskManager.Task::(_decref) Task=`717547e7-8f58-4366-ad9a-786555254793`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:58:45,957::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:58:46,823::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:58:46,898::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317688 s, 32.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:58:48,910::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:58:48,976::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:58:48,988::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0324262 s, 31.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:58:51,001::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:58:51,080::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355663 s, 28.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:58:51,996::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:58:52,715::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:58:52,723::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000860101 s, 4.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:58:53,093::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:58:53,462::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.327472 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:58:55,016::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:58:55,461::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:58:55,462::task::595::TaskManager.Task::(_updateState) Task=`41ac2a7d-6da7-4a20-afcb-d35960d0554f`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:58:55,462::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:58:55,462::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:58:55,462::task::1185::TaskManager.Task::(prepare) Task=`41ac2a7d-6da7-4a20-afcb-d35960d0554f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:58:55,463::task::595::TaskManager.Task::(_updateState) Task=`41ac2a7d-6da7-4a20-afcb-d35960d0554f`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:58:55,463::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:58:55,463::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:58:55,463::task::990::TaskManager.Task::(_decref) Task=`41ac2a7d-6da7-4a20-afcb-d35960d0554f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:58:55,473::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:58:55,473::task::595::TaskManager.Task::(_updateState) Task=`5ebacdd2-7bc3-4171-89ce-ac2933b1c95e`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:58:55,473::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:58:55,474::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3932f18d-52e4-4c16-ad77-0dbbc07c0cca`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:58:55,474::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:58:55,474::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:58:55,474::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3932f18d-52e4-4c16-ad77-0dbbc07c0cca`::Granted request >Thread-338::DEBUG::2014-05-13 11:58:55,475::task::827::TaskManager.Task::(resourceAcquired) Task=`5ebacdd2-7bc3-4171-89ce-ac2933b1c95e`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:58:55,475::task::990::TaskManager.Task::(_decref) Task=`5ebacdd2-7bc3-4171-89ce-ac2933b1c95e`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:58:55,477::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Dummy-337::DEBUG::2014-05-13 11:58:55,477::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-338::DEBUG::2014-05-13 11:58:55,478::task::1185::TaskManager.Task::(prepare) Task=`5ebacdd2-7bc3-4171-89ce-ac2933b1c95e`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:58:55,487::task::595::TaskManager.Task::(_updateState) Task=`5ebacdd2-7bc3-4171-89ce-ac2933b1c95e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:58:55,489::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:58:55,490::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:58:55,493::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:58:55,495::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:58:55,496::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:58:55,498::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:58:55,501::task::990::TaskManager.Task::(_decref) Task=`5ebacdd2-7bc3-4171-89ce-ac2933b1c95e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:58:55,568::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0478565 s, 21.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:58:57,581::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:58:57,655::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0367012 s, 27.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:58:58,035::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:58:59,663::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:58:59,735::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305321 s, 33.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:59:00,891::task::595::TaskManager.Task::(_updateState) Task=`39df2dcf-eb4e-4105-8b7d-f1fbe905e5ee`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:59:00,891::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:59:00,892::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000860101', 'lastCheck': '8.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:59:00,892::task::1185::TaskManager.Task::(prepare) Task=`39df2dcf-eb4e-4105-8b7d-f1fbe905e5ee`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000860101', 'lastCheck': '8.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:59:00,892::task::595::TaskManager.Task::(_updateState) Task=`39df2dcf-eb4e-4105-8b7d-f1fbe905e5ee`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:59:00,892::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:59:00,892::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:59:00,892::task::990::TaskManager.Task::(_decref) Task=`39df2dcf-eb4e-4105-8b7d-f1fbe905e5ee`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:59:01,056::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:59:01,748::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:59:01,824::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320078 s, 32.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:59:02,731::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:59:02,739::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000677955 s, 6.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:59:03,837::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:59:03,912::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316796 s, 32.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:59:04,075::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:59:05,527::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:59:05,529::task::595::TaskManager.Task::(_updateState) Task=`0af385c2-6024-47e1-b41e-fbb6775311f3`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:59:05,530::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:59:05,531::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:59:05,532::task::1185::TaskManager.Task::(prepare) Task=`0af385c2-6024-47e1-b41e-fbb6775311f3`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:59:05,533::task::595::TaskManager.Task::(_updateState) Task=`0af385c2-6024-47e1-b41e-fbb6775311f3`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:59:05,533::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:59:05,534::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:59:05,534::task::990::TaskManager.Task::(_decref) Task=`0af385c2-6024-47e1-b41e-fbb6775311f3`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:59:05,539::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:59:05,539::task::595::TaskManager.Task::(_updateState) Task=`9ba3fe3f-27a6-4bcd-9c55-b675d6021ae3`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:59:05,539::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:59:05,540::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c8bbe364-9af6-4174-a1f5-238ed15afdcf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:59:05,540::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:59:05,540::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:59:05,540::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c8bbe364-9af6-4174-a1f5-238ed15afdcf`::Granted request >Thread-338::DEBUG::2014-05-13 11:59:05,542::task::827::TaskManager.Task::(resourceAcquired) Task=`9ba3fe3f-27a6-4bcd-9c55-b675d6021ae3`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:59:05,542::task::990::TaskManager.Task::(_decref) Task=`9ba3fe3f-27a6-4bcd-9c55-b675d6021ae3`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:59:05,542::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:59:05,543::task::1185::TaskManager.Task::(prepare) Task=`9ba3fe3f-27a6-4bcd-9c55-b675d6021ae3`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:59:05,543::task::595::TaskManager.Task::(_updateState) Task=`9ba3fe3f-27a6-4bcd-9c55-b675d6021ae3`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:59:05,543::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:59:05,543::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:59:05,543::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:59:05,544::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:59:05,544::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:59:05,544::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:59:05,544::task::990::TaskManager.Task::(_decref) Task=`9ba3fe3f-27a6-4bcd-9c55-b675d6021ae3`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:59:05,925::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:59:06,003::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036578 s, 28.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:59:07,093::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:59:08,017::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:59:08,104::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0427461 s, 24.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:59:10,111::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:59:10,117::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:59:10,196::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310839 s, 32.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:59:12,211::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:59:12,290::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353025 s, 29.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:59:12,746::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:59:12,753::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000604623 s, 6.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:59:13,132::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:59:14,303::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:59:14,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381534 s, 26.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:59:15,562::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [206acf2a] >Thread-66::DEBUG::2014-05-13 11:59:15,563::task::595::TaskManager.Task::(_updateState) Task=`7e96a9a9-8726-45b3-a2e1-896a69cb3c79`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:59:15,563::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:59:15,564::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:59:15,564::task::1185::TaskManager.Task::(prepare) Task=`7e96a9a9-8726-45b3-a2e1-896a69cb3c79`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:59:15,564::task::595::TaskManager.Task::(_updateState) Task=`7e96a9a9-8726-45b3-a2e1-896a69cb3c79`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:59:15,564::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:59:15,564::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:59:15,564::task::990::TaskManager.Task::(_decref) Task=`7e96a9a9-8726-45b3-a2e1-896a69cb3c79`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:59:15,569::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [206acf2a] >Thread-338::DEBUG::2014-05-13 11:59:15,569::task::595::TaskManager.Task::(_updateState) Task=`78e16520-8fe0-422c-be8c-01e075ad935d`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:59:15,570::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:59:15,570::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`98b89e2e-1965-4a42-8e53-168ff821d66a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:59:15,570::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:59:15,571::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:59:15,571::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`98b89e2e-1965-4a42-8e53-168ff821d66a`::Granted request >Thread-338::DEBUG::2014-05-13 11:59:15,571::task::827::TaskManager.Task::(resourceAcquired) Task=`78e16520-8fe0-422c-be8c-01e075ad935d`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:59:15,572::task::990::TaskManager.Task::(_decref) Task=`78e16520-8fe0-422c-be8c-01e075ad935d`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:59:15,572::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:59:15,572::task::1185::TaskManager.Task::(prepare) Task=`78e16520-8fe0-422c-be8c-01e075ad935d`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:59:15,573::task::595::TaskManager.Task::(_updateState) Task=`78e16520-8fe0-422c-be8c-01e075ad935d`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:59:15,573::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:59:15,573::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:59:15,574::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:59:15,574::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:59:15,574::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:59:15,574::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:59:15,574::task::990::TaskManager.Task::(_decref) Task=`78e16520-8fe0-422c-be8c-01e075ad935d`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 11:59:16,024::task::595::TaskManager.Task::(_updateState) Task=`040b4225-053f-4cbc-b642-ac00cf57b0d2`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:59:16,024::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:59:16,024::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000604623', 'lastCheck': '3.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:59:16,024::task::1185::TaskManager.Task::(prepare) Task=`040b4225-053f-4cbc-b642-ac00cf57b0d2`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000604623', 'lastCheck': '3.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:59:16,024::task::595::TaskManager.Task::(_updateState) Task=`040b4225-053f-4cbc-b642-ac00cf57b0d2`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:59:16,024::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:59:16,025::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:59:16,025::task::990::TaskManager.Task::(_decref) Task=`040b4225-053f-4cbc-b642-ac00cf57b0d2`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:59:16,151::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:59:16,399::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:59:16,473::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031498 s, 32.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:59:18,486::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:59:18,569::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0380814 s, 26.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:59:19,169::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:59:20,583::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:59:20,668::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0388395 s, 26.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:59:22,191::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:59:22,681::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 11:59:22,780::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:59:22,798::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000657461 s, 6.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:59:22,827::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0433398 s, 23.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:59:24,840::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:59:24,914::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307663 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:59:25,209::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:59:25,592::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17df19a0] >Thread-66::DEBUG::2014-05-13 11:59:25,592::task::595::TaskManager.Task::(_updateState) Task=`732d7e06-3bfc-4ea7-93f3-d3617c9cf5f1`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:59:25,592::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:59:25,593::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:59:25,593::task::1185::TaskManager.Task::(prepare) Task=`732d7e06-3bfc-4ea7-93f3-d3617c9cf5f1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:59:25,593::task::595::TaskManager.Task::(_updateState) Task=`732d7e06-3bfc-4ea7-93f3-d3617c9cf5f1`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:59:25,593::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:59:25,594::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:59:25,594::task::990::TaskManager.Task::(_decref) Task=`732d7e06-3bfc-4ea7-93f3-d3617c9cf5f1`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:59:25,599::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17df19a0] >Thread-338::DEBUG::2014-05-13 11:59:25,599::task::595::TaskManager.Task::(_updateState) Task=`a7831516-f3fd-4632-9a27-2546460241eb`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:59:25,599::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:59:25,600::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9b7f7f64-e022-4b64-a409-aac798b6727c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:59:25,600::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:59:25,600::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:59:25,600::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9b7f7f64-e022-4b64-a409-aac798b6727c`::Granted request >Thread-338::DEBUG::2014-05-13 11:59:25,601::task::827::TaskManager.Task::(resourceAcquired) Task=`a7831516-f3fd-4632-9a27-2546460241eb`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:59:25,601::task::990::TaskManager.Task::(_decref) Task=`a7831516-f3fd-4632-9a27-2546460241eb`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:59:25,602::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:59:25,602::task::1185::TaskManager.Task::(prepare) Task=`a7831516-f3fd-4632-9a27-2546460241eb`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:59:25,602::task::595::TaskManager.Task::(_updateState) Task=`a7831516-f3fd-4632-9a27-2546460241eb`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:59:25,602::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:59:25,602::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:59:25,603::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:59:25,603::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:59:25,603::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:59:25,603::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:59:25,603::task::990::TaskManager.Task::(_decref) Task=`a7831516-f3fd-4632-9a27-2546460241eb`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:59:26,927::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:59:27,009::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0392159 s, 26.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:59:28,229::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:59:29,022::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:59:29,099::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0326493 s, 31.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:59:31,112::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 11:59:31,163::task::595::TaskManager.Task::(_updateState) Task=`3b83b825-27f4-41ac-9e26-7b392456a963`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:59:31,164::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:59:31,165::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000657461', 'lastCheck': '8.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:59:31,166::task::1185::TaskManager.Task::(prepare) Task=`3b83b825-27f4-41ac-9e26-7b392456a963`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000657461', 'lastCheck': '8.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:59:31,167::task::595::TaskManager.Task::(_updateState) Task=`3b83b825-27f4-41ac-9e26-7b392456a963`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:59:31,168::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:59:31,169::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:59:31,170::task::990::TaskManager.Task::(_decref) Task=`3b83b825-27f4-41ac-9e26-7b392456a963`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:59:31,215::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319969 s, 32.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:59:31,248::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 11:59:32,823::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:59:32,830::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000658193 s, 6.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:59:33,228::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:59:33,309::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0388877 s, 26.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:59:34,268::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:59:35,322::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:59:35,402::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0377522 s, 27.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:59:35,622::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2555e502] >Thread-66::DEBUG::2014-05-13 11:59:35,623::task::595::TaskManager.Task::(_updateState) Task=`11e98f13-6ed2-4304-9835-5428d6074af3`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:59:35,623::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:59:35,623::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:59:35,623::task::1185::TaskManager.Task::(prepare) Task=`11e98f13-6ed2-4304-9835-5428d6074af3`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:59:35,624::task::595::TaskManager.Task::(_updateState) Task=`11e98f13-6ed2-4304-9835-5428d6074af3`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:59:35,624::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:59:35,624::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:59:35,624::task::990::TaskManager.Task::(_decref) Task=`11e98f13-6ed2-4304-9835-5428d6074af3`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:59:35,629::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2555e502] >Thread-338::DEBUG::2014-05-13 11:59:35,629::task::595::TaskManager.Task::(_updateState) Task=`c3d290c9-84c8-4fbb-812a-feeb17aa9aab`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:59:35,629::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:59:35,630::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`12e124aa-81f4-4963-b917-a526b8d78d29`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:59:35,630::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:59:35,630::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:59:35,630::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`12e124aa-81f4-4963-b917-a526b8d78d29`::Granted request >Thread-338::DEBUG::2014-05-13 11:59:35,631::task::827::TaskManager.Task::(resourceAcquired) Task=`c3d290c9-84c8-4fbb-812a-feeb17aa9aab`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:59:35,631::task::990::TaskManager.Task::(_decref) Task=`c3d290c9-84c8-4fbb-812a-feeb17aa9aab`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:59:35,632::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:59:35,632::task::1185::TaskManager.Task::(prepare) Task=`c3d290c9-84c8-4fbb-812a-feeb17aa9aab`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:59:35,632::task::595::TaskManager.Task::(_updateState) Task=`c3d290c9-84c8-4fbb-812a-feeb17aa9aab`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:59:35,632::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:59:35,632::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:59:35,633::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:59:35,633::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:59:35,633::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:59:35,633::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:59:35,633::task::990::TaskManager.Task::(_decref) Task=`c3d290c9-84c8-4fbb-812a-feeb17aa9aab`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:59:37,289::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:59:37,415::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:59:37,783::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.326024 s, 3.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:59:39,796::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:59:39,871::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318734 s, 32.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:59:40,305::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:59:41,884::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:59:41,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357354 s, 28.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:59:42,837::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:59:42,845::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000705808 s, 5.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:59:43,325::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:59:43,977::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:59:44,059::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0388763 s, 26.3 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:59:45,651::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 11:59:45,651::task::595::TaskManager.Task::(_updateState) Task=`6e322eb0-27b0-4ce9-b1e5-92010b43012e`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:59:45,651::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:59:45,652::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:59:45,652::task::1185::TaskManager.Task::(prepare) Task=`6e322eb0-27b0-4ce9-b1e5-92010b43012e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:59:45,652::task::595::TaskManager.Task::(_updateState) Task=`6e322eb0-27b0-4ce9-b1e5-92010b43012e`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:59:45,652::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:59:45,653::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:59:45,653::task::990::TaskManager.Task::(_decref) Task=`6e322eb0-27b0-4ce9-b1e5-92010b43012e`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:59:45,657::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 11:59:45,657::task::595::TaskManager.Task::(_updateState) Task=`9bd1d4f5-b620-47a9-a466-074ac1167e3d`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:59:45,658::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:59:45,658::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4fcb3168-4718-426a-9569-ca4bcf7fcbdb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:59:45,658::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:59:45,659::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:59:45,659::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4fcb3168-4718-426a-9569-ca4bcf7fcbdb`::Granted request >Thread-338::DEBUG::2014-05-13 11:59:45,659::task::827::TaskManager.Task::(resourceAcquired) Task=`9bd1d4f5-b620-47a9-a466-074ac1167e3d`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:59:45,659::task::990::TaskManager.Task::(_decref) Task=`9bd1d4f5-b620-47a9-a466-074ac1167e3d`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:59:45,660::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:59:45,660::task::1185::TaskManager.Task::(prepare) Task=`9bd1d4f5-b620-47a9-a466-074ac1167e3d`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:59:45,660::task::595::TaskManager.Task::(_updateState) Task=`9bd1d4f5-b620-47a9-a466-074ac1167e3d`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:59:45,661::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:59:45,661::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:59:45,661::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:59:45,661::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:59:45,661::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:59:45,661::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:59:45,662::task::990::TaskManager.Task::(_decref) Task=`9bd1d4f5-b620-47a9-a466-074ac1167e3d`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:59:46,072::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:59:46,152::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361105 s, 28.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 11:59:46,315::task::595::TaskManager.Task::(_updateState) Task=`95482ea3-9fd8-4a6b-b649-e5f22daecaa8`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:59:46,315::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 11:59:46,315::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000705808', 'lastCheck': '3.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:59:46,316::task::1185::TaskManager.Task::(prepare) Task=`95482ea3-9fd8-4a6b-b649-e5f22daecaa8`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000705808', 'lastCheck': '3.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 11:59:46,316::task::595::TaskManager.Task::(_updateState) Task=`95482ea3-9fd8-4a6b-b649-e5f22daecaa8`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:59:46,316::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:59:46,316::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:59:46,317::task::990::TaskManager.Task::(_decref) Task=`95482ea3-9fd8-4a6b-b649-e5f22daecaa8`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 11:59:46,351::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:59:48,165::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:59:48,255::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0463132 s, 22.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:59:49,370::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:59:50,267::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:59:50,342::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306091 s, 33.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:59:52,358::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 11:59:52,391::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:59:52,439::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0324193 s, 31.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 11:59:52,853::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 11:59:52,861::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000745879 s, 5.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 11:59:54,451::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:59:54,532::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0371871 s, 27.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:59:55,410::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 11:59:55,680::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [7a000b17] >Thread-66::DEBUG::2014-05-13 11:59:55,681::task::595::TaskManager.Task::(_updateState) Task=`f9906edb-0fb0-4174-9bbb-8464368ce350`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 11:59:55,681::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 11:59:55,681::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:59:55,682::task::1185::TaskManager.Task::(prepare) Task=`f9906edb-0fb0-4174-9bbb-8464368ce350`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 11:59:55,682::task::595::TaskManager.Task::(_updateState) Task=`f9906edb-0fb0-4174-9bbb-8464368ce350`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 11:59:55,682::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 11:59:55,682::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 11:59:55,682::task::990::TaskManager.Task::(_decref) Task=`f9906edb-0fb0-4174-9bbb-8464368ce350`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 11:59:55,687::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [7a000b17] >Thread-338::DEBUG::2014-05-13 11:59:55,687::task::595::TaskManager.Task::(_updateState) Task=`647fcb4c-2062-4083-a5ca-408208cddb47`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 11:59:55,688::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 11:59:55,688::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`94e95b56-8ca4-4cad-a399-7f341e5dfe3f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 11:59:55,688::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 11:59:55,688::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 11:59:55,689::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`94e95b56-8ca4-4cad-a399-7f341e5dfe3f`::Granted request >Thread-338::DEBUG::2014-05-13 11:59:55,689::task::827::TaskManager.Task::(resourceAcquired) Task=`647fcb4c-2062-4083-a5ca-408208cddb47`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 11:59:55,689::task::990::TaskManager.Task::(_decref) Task=`647fcb4c-2062-4083-a5ca-408208cddb47`::ref 1 aborting False >Thread-338::INFO::2014-05-13 11:59:55,690::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:59:55,690::task::1185::TaskManager.Task::(prepare) Task=`647fcb4c-2062-4083-a5ca-408208cddb47`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 11:59:55,690::task::595::TaskManager.Task::(_updateState) Task=`647fcb4c-2062-4083-a5ca-408208cddb47`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 11:59:55,690::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 11:59:55,691::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 11:59:55,691::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 11:59:55,691::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 11:59:55,691::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 11:59:55,691::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 11:59:55,692::task::990::TaskManager.Task::(_decref) Task=`647fcb4c-2062-4083-a5ca-408208cddb47`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 11:59:56,545::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:59:56,624::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364065 s, 28.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 11:59:58,431::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 11:59:58,637::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 11:59:58,713::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321977 s, 31.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:00:00,726::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:00,801::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031094 s, 32.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:00:01,449::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:00:01,452::task::595::TaskManager.Task::(_updateState) Task=`e12a9d99-403f-4509-96d5-3d541f5302ab`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:00:01,452::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:00:01,452::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000745879', 'lastCheck': '8.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:00:01,453::task::1185::TaskManager.Task::(prepare) Task=`e12a9d99-403f-4509-96d5-3d541f5302ab`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000745879', 'lastCheck': '8.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:00:01,453::task::595::TaskManager.Task::(_updateState) Task=`e12a9d99-403f-4509-96d5-3d541f5302ab`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:00:01,453::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:00:01,453::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:00:01,453::task::990::TaskManager.Task::(_decref) Task=`e12a9d99-403f-4509-96d5-3d541f5302ab`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:00:02,814::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 12:00:02,866::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 41830dad-f012-4b4d-b636-863f66d5ca4b >Thread-322::DEBUG::2014-05-13 12:00:02,868::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-322::DEBUG::2014-05-13 12:00:02,890::blockSD::336::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:00:02,906::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000575384 s, 3.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:00:02,908::misc::287::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000575384 s, 3.6 MB/s'], size: 2048 >Thread-322::DEBUG::2014-05-13 12:00:02,909::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-322::WARNING::2014-05-13 12:00:02,910::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-322::DEBUG::2014-05-13 12:00:02,911::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-322::DEBUG::2014-05-13 12:00:02,913::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:00:02,915::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:00:02,918::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:00:02,920::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:00:02,921::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:00:02,932::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:02,979::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0994478 s, 10.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:00:03,148::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:00:03,149::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:00:03,150::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'LOCKPOLICY=', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', 'DESCRIPTION=ISCSI', 'LOGBLKSIZE=512', 'TYPE=ISCSI', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'PHYBLKSIZE=512', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', 'CLASS=Data', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'POOL_DESCRIPTION=DC_2', 'POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'MASTER_VERSION=1', 'ROLE=Master', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', '_SHA_CKSUM=28bb2d7cc60e0d779eb69fe93d050c16c769f069', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0'] >Thread-322::INFO::2014-05-13 12:00:03,151::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_imageNS already registered >Thread-322::INFO::2014-05-13 12:00:03,151::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_volumeNS already registered >Thread-322::INFO::2014-05-13 12:00:03,151::blockSD::452::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_lvmActivationNS already registered >Thread-322::DEBUG::2014-05-13 12:00:03,152::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 12:00:03,321::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:00:03,324::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:00:03,333::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000696875 s, 5.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:00:04,468::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:00:04,992::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:05,068::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031503 s, 32.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:00:05,710::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:00:05,710::task::595::TaskManager.Task::(_updateState) Task=`e981b1f9-9499-4d49-a243-dd33f9f7499a`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:00:05,710::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:00:05,711::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:00:05,711::task::1185::TaskManager.Task::(prepare) Task=`e981b1f9-9499-4d49-a243-dd33f9f7499a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:00:05,711::task::595::TaskManager.Task::(_updateState) Task=`e981b1f9-9499-4d49-a243-dd33f9f7499a`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:00:05,711::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:00:05,711::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:00:05,712::task::990::TaskManager.Task::(_decref) Task=`e981b1f9-9499-4d49-a243-dd33f9f7499a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:00:05,716::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:00:05,717::task::595::TaskManager.Task::(_updateState) Task=`104d3d51-7a20-410e-8219-37936c7510f2`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:00:05,717::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:00:05,717::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e63a62ad-7b82-4878-879f-f2f6140a92b9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:00:05,717::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:00:05,718::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:00:05,718::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e63a62ad-7b82-4878-879f-f2f6140a92b9`::Granted request >Thread-338::DEBUG::2014-05-13 12:00:05,718::task::827::TaskManager.Task::(resourceAcquired) Task=`104d3d51-7a20-410e-8219-37936c7510f2`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:00:05,718::task::990::TaskManager.Task::(_decref) Task=`104d3d51-7a20-410e-8219-37936c7510f2`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:00:05,719::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:00:05,719::task::1185::TaskManager.Task::(prepare) Task=`104d3d51-7a20-410e-8219-37936c7510f2`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:00:05,720::task::595::TaskManager.Task::(_updateState) Task=`104d3d51-7a20-410e-8219-37936c7510f2`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:00:05,720::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:00:05,720::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:00:05,720::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:00:05,720::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:00:05,721::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:00:05,721::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:00:05,721::task::990::TaskManager.Task::(_decref) Task=`104d3d51-7a20-410e-8219-37936c7510f2`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:00:07,081::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:07,157::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0325146 s, 31.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:00:07,488::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:00:09,170::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:09,253::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382005 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:00:10,523::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:00:11,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:11,345::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035888 s, 28.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:00:13,339::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:00:13,347::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000717564 s, 5.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:00:13,357::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:13,436::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363672 s, 28.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:00:13,543::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:00:15,449::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:15,542::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0509544 s, 20.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:00:15,738::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3179350e] >Thread-66::DEBUG::2014-05-13 12:00:15,738::task::595::TaskManager.Task::(_updateState) Task=`51d40674-c308-4878-8b2f-597e83e25544`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:00:15,739::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:00:15,739::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:00:15,739::task::1185::TaskManager.Task::(prepare) Task=`51d40674-c308-4878-8b2f-597e83e25544`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:00:15,739::task::595::TaskManager.Task::(_updateState) Task=`51d40674-c308-4878-8b2f-597e83e25544`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:00:15,739::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:00:15,740::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:00:15,740::task::990::TaskManager.Task::(_decref) Task=`51d40674-c308-4878-8b2f-597e83e25544`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:00:15,744::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3179350e] >Thread-338::DEBUG::2014-05-13 12:00:15,744::task::595::TaskManager.Task::(_updateState) Task=`58eee51c-c34f-405d-8d65-13f6c1e5d019`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:00:15,745::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:00:15,745::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ebe04530-29f0-4929-a008-f06993a06ec8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:00:15,745::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:00:15,745::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:00:15,745::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ebe04530-29f0-4929-a008-f06993a06ec8`::Granted request >Thread-338::DEBUG::2014-05-13 12:00:15,746::task::827::TaskManager.Task::(resourceAcquired) Task=`58eee51c-c34f-405d-8d65-13f6c1e5d019`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:00:15,746::task::990::TaskManager.Task::(_decref) Task=`58eee51c-c34f-405d-8d65-13f6c1e5d019`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:00:15,747::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:00:15,747::task::1185::TaskManager.Task::(prepare) Task=`58eee51c-c34f-405d-8d65-13f6c1e5d019`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:00:15,747::task::595::TaskManager.Task::(_updateState) Task=`58eee51c-c34f-405d-8d65-13f6c1e5d019`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:00:15,747::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:00:15,747::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:00:15,747::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:00:15,748::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:00:15,748::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:00:15,748::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:00:15,748::task::990::TaskManager.Task::(_decref) Task=`58eee51c-c34f-405d-8d65-13f6c1e5d019`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:00:16,562::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:00:16,586::task::595::TaskManager.Task::(_updateState) Task=`0da1815a-1df5-4215-9aff-27a26caba053`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:00:16,586::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:00:16,586::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000717564', 'lastCheck': '3.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:00:16,586::task::1185::TaskManager.Task::(prepare) Task=`0da1815a-1df5-4215-9aff-27a26caba053`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000717564', 'lastCheck': '3.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:00:16,586::task::595::TaskManager.Task::(_updateState) Task=`0da1815a-1df5-4215-9aff-27a26caba053`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:00:16,587::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:00:16,587::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:00:16,587::task::990::TaskManager.Task::(_decref) Task=`0da1815a-1df5-4215-9aff-27a26caba053`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:00:17,555::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:17,636::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0388623 s, 26.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:00:19,589::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:00:19,649::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:19,743::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374764 s, 27.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:00:21,756::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:21,831::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311542 s, 32.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:00:22,608::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:00:23,355::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:00:23,363::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000793116 s, 5.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:00:23,844::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:23,925::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381553 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:00:25,630::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:00:25,772::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:00:25,772::task::595::TaskManager.Task::(_updateState) Task=`8b05c59b-727d-46c5-b819-3b80cc0792ab`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:00:25,772::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:00:25,773::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:00:25,773::task::1185::TaskManager.Task::(prepare) Task=`8b05c59b-727d-46c5-b819-3b80cc0792ab`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:00:25,773::task::595::TaskManager.Task::(_updateState) Task=`8b05c59b-727d-46c5-b819-3b80cc0792ab`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:00:25,773::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:00:25,773::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:00:25,773::task::990::TaskManager.Task::(_decref) Task=`8b05c59b-727d-46c5-b819-3b80cc0792ab`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:00:25,778::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:00:25,778::task::595::TaskManager.Task::(_updateState) Task=`c6305a01-9b71-4d6e-9280-3212eccbb3bf`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:00:25,779::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:00:25,779::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9d4cd05d-177d-42d1-8028-2b36c701053f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:00:25,779::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:00:25,780::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:00:25,780::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9d4cd05d-177d-42d1-8028-2b36c701053f`::Granted request >Thread-338::DEBUG::2014-05-13 12:00:25,780::task::827::TaskManager.Task::(resourceAcquired) Task=`c6305a01-9b71-4d6e-9280-3212eccbb3bf`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:00:25,780::task::990::TaskManager.Task::(_decref) Task=`c6305a01-9b71-4d6e-9280-3212eccbb3bf`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:00:25,781::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:00:25,781::task::1185::TaskManager.Task::(prepare) Task=`c6305a01-9b71-4d6e-9280-3212eccbb3bf`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:00:25,782::task::595::TaskManager.Task::(_updateState) Task=`c6305a01-9b71-4d6e-9280-3212eccbb3bf`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:00:25,782::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:00:25,782::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:00:25,782::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:00:25,782::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:00:25,782::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:00:25,783::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:00:25,783::task::990::TaskManager.Task::(_decref) Task=`c6305a01-9b71-4d6e-9280-3212eccbb3bf`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:00:25,938::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:26,021::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383605 s, 26.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:00:28,034::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:28,109::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319353 s, 32.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:00:28,649::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:00:30,122::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:30,494::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.329383 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:00:31,666::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:00:31,718::task::595::TaskManager.Task::(_updateState) Task=`6626ec9e-2192-4b83-bd8a-b80bfb71c7d6`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:00:31,718::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:00:31,719::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000793116', 'lastCheck': '8.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:00:31,719::task::1185::TaskManager.Task::(prepare) Task=`6626ec9e-2192-4b83-bd8a-b80bfb71c7d6`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000793116', 'lastCheck': '8.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:00:31,719::task::595::TaskManager.Task::(_updateState) Task=`6626ec9e-2192-4b83-bd8a-b80bfb71c7d6`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:00:31,719::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:00:31,719::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:00:31,719::task::990::TaskManager.Task::(_decref) Task=`6626ec9e-2192-4b83-bd8a-b80bfb71c7d6`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:00:32,507::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:32,585::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362855 s, 28.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:00:33,370::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:00:33,378::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000585881 s, 7.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:00:34,598::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:34,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0327647 s, 31.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:00:34,684::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:00:35,800::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:00:35,800::task::595::TaskManager.Task::(_updateState) Task=`bda28fcd-4d92-4288-b7a3-9ed1a9e84063`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:00:35,801::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:00:35,801::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:00:35,801::task::1185::TaskManager.Task::(prepare) Task=`bda28fcd-4d92-4288-b7a3-9ed1a9e84063`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:00:35,801::task::595::TaskManager.Task::(_updateState) Task=`bda28fcd-4d92-4288-b7a3-9ed1a9e84063`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:00:35,801::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:00:35,802::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:00:35,802::task::990::TaskManager.Task::(_decref) Task=`bda28fcd-4d92-4288-b7a3-9ed1a9e84063`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:00:35,806::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:00:35,807::task::595::TaskManager.Task::(_updateState) Task=`82fcbfdd-58fd-437b-9796-36a5061306f9`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:00:35,807::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:00:35,807::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1e01f50e-660a-4c7c-a0e5-aed7ca0e31c9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:00:35,807::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:00:35,808::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:00:35,808::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1e01f50e-660a-4c7c-a0e5-aed7ca0e31c9`::Granted request >Thread-338::DEBUG::2014-05-13 12:00:35,808::task::827::TaskManager.Task::(resourceAcquired) Task=`82fcbfdd-58fd-437b-9796-36a5061306f9`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:00:35,808::task::990::TaskManager.Task::(_decref) Task=`82fcbfdd-58fd-437b-9796-36a5061306f9`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:00:35,809::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:00:35,809::task::1185::TaskManager.Task::(prepare) Task=`82fcbfdd-58fd-437b-9796-36a5061306f9`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:00:35,810::task::595::TaskManager.Task::(_updateState) Task=`82fcbfdd-58fd-437b-9796-36a5061306f9`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:00:35,810::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:00:35,810::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:00:35,810::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:00:35,810::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:00:35,810::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:00:35,811::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:00:35,811::task::990::TaskManager.Task::(_decref) Task=`82fcbfdd-58fd-437b-9796-36a5061306f9`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:00:36,687::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:36,760::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036588 s, 28.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:00:37,704::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:00:38,768::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:38,844::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319383 s, 32.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:00:40,722::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:00:40,857::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:40,937::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362234 s, 28.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:00:42,950::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:43,024::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0328913 s, 31.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:00:43,385::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:00:43,393::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000672556 s, 6.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:00:43,741::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:00:45,037::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:45,128::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0468676 s, 21.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:00:45,829::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:00:45,830::task::595::TaskManager.Task::(_updateState) Task=`3375f5da-6f70-4b23-b8a4-1da7bd839693`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:00:45,830::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:00:45,830::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:00:45,830::task::1185::TaskManager.Task::(prepare) Task=`3375f5da-6f70-4b23-b8a4-1da7bd839693`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:00:45,830::task::595::TaskManager.Task::(_updateState) Task=`3375f5da-6f70-4b23-b8a4-1da7bd839693`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:00:45,831::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:00:45,831::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:00:45,831::task::990::TaskManager.Task::(_decref) Task=`3375f5da-6f70-4b23-b8a4-1da7bd839693`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:00:45,835::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:00:45,835::task::595::TaskManager.Task::(_updateState) Task=`5a8b21c4-b22d-40ad-91b9-607465c700c4`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:00:45,836::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:00:45,836::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a9270cc3-5f1f-4aaf-8013-f390faa30138`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:00:45,836::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:00:45,836::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:00:45,837::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a9270cc3-5f1f-4aaf-8013-f390faa30138`::Granted request >Thread-338::DEBUG::2014-05-13 12:00:45,837::task::827::TaskManager.Task::(resourceAcquired) Task=`5a8b21c4-b22d-40ad-91b9-607465c700c4`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:00:45,837::task::990::TaskManager.Task::(_decref) Task=`5a8b21c4-b22d-40ad-91b9-607465c700c4`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:00:45,838::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:00:45,838::task::1185::TaskManager.Task::(prepare) Task=`5a8b21c4-b22d-40ad-91b9-607465c700c4`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:00:45,838::task::595::TaskManager.Task::(_updateState) Task=`5a8b21c4-b22d-40ad-91b9-607465c700c4`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:00:45,838::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:00:45,838::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:00:45,839::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:00:45,839::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:00:45,839::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:00:45,839::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:00:45,839::task::990::TaskManager.Task::(_decref) Task=`5a8b21c4-b22d-40ad-91b9-607465c700c4`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:00:46,761::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:00:46,855::task::595::TaskManager.Task::(_updateState) Task=`1f6c0cee-831a-4c5c-bbfc-36d85418fda0`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:00:46,855::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:00:46,855::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000672556', 'lastCheck': '3.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:00:46,855::task::1185::TaskManager.Task::(prepare) Task=`1f6c0cee-831a-4c5c-bbfc-36d85418fda0`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000672556', 'lastCheck': '3.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:00:46,855::task::595::TaskManager.Task::(_updateState) Task=`1f6c0cee-831a-4c5c-bbfc-36d85418fda0`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:00:46,855::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:00:46,856::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:00:46,856::task::990::TaskManager.Task::(_decref) Task=`1f6c0cee-831a-4c5c-bbfc-36d85418fda0`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:00:47,142::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:47,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0325388 s, 31.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:00:49,230::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:49,305::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311635 s, 32.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:00:49,778::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:00:51,318::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:51,401::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0380143 s, 26.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:00:52,795::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:00:53,400::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:00:53,408::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000704848 s, 5.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:00:53,413::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:53,491::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0351269 s, 29.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:00:55,503::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:55,578::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319003 s, 32.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:00:55,814::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:00:55,857::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17df19a0] >Thread-66::DEBUG::2014-05-13 12:00:55,858::task::595::TaskManager.Task::(_updateState) Task=`b4c2eb61-45bf-476f-9c94-86a4e09cca2b`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:00:55,858::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:00:55,858::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:00:55,859::task::1185::TaskManager.Task::(prepare) Task=`b4c2eb61-45bf-476f-9c94-86a4e09cca2b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:00:55,859::task::595::TaskManager.Task::(_updateState) Task=`b4c2eb61-45bf-476f-9c94-86a4e09cca2b`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:00:55,859::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:00:55,859::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:00:55,859::task::990::TaskManager.Task::(_decref) Task=`b4c2eb61-45bf-476f-9c94-86a4e09cca2b`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:00:55,864::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17df19a0] >Thread-338::DEBUG::2014-05-13 12:00:55,864::task::595::TaskManager.Task::(_updateState) Task=`930169e6-cb19-43e7-acc1-dc8d003a8488`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:00:55,864::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:00:55,864::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e6e80e1e-26d1-43cd-9077-8020a553a0ea`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:00:55,865::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:00:55,865::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:00:55,865::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e6e80e1e-26d1-43cd-9077-8020a553a0ea`::Granted request >Thread-338::DEBUG::2014-05-13 12:00:55,866::task::827::TaskManager.Task::(resourceAcquired) Task=`930169e6-cb19-43e7-acc1-dc8d003a8488`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:00:55,866::task::990::TaskManager.Task::(_decref) Task=`930169e6-cb19-43e7-acc1-dc8d003a8488`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:00:55,866::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:00:55,867::task::1185::TaskManager.Task::(prepare) Task=`930169e6-cb19-43e7-acc1-dc8d003a8488`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:00:55,867::task::595::TaskManager.Task::(_updateState) Task=`930169e6-cb19-43e7-acc1-dc8d003a8488`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:00:55,867::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:00:55,867::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:00:55,867::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:00:55,868::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:00:55,868::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:00:55,868::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:00:55,868::task::990::TaskManager.Task::(_decref) Task=`930169e6-cb19-43e7-acc1-dc8d003a8488`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:00:57,591::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:00:57,955::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.321408 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:00:58,838::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:00:59,968::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:00,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363433 s, 28.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:01:01,855::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:01:01,981::task::595::TaskManager.Task::(_updateState) Task=`3f97f882-c5dc-4a05-8958-2e36ec384c6d`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:01:01,981::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:01:01,981::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000704848', 'lastCheck': '8.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:01:01,981::task::1185::TaskManager.Task::(prepare) Task=`3f97f882-c5dc-4a05-8958-2e36ec384c6d`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000704848', 'lastCheck': '8.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:01:01,981::task::595::TaskManager.Task::(_updateState) Task=`3f97f882-c5dc-4a05-8958-2e36ec384c6d`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:01:01,982::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:01:01,982::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:01:01,982::task::990::TaskManager.Task::(_decref) Task=`3f97f882-c5dc-4a05-8958-2e36ec384c6d`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:01:02,061::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:02,137::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318172 s, 32.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:01:03,415::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:01:03,447::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00071011 s, 5.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:01:04,150::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:04,225::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309838 s, 33.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:01:04,875::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:01:05,887::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:01:05,888::task::595::TaskManager.Task::(_updateState) Task=`8a7cdef4-3793-4dd2-aafa-023c78255461`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:01:05,888::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:01:05,888::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:01:05,888::task::1185::TaskManager.Task::(prepare) Task=`8a7cdef4-3793-4dd2-aafa-023c78255461`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:01:05,889::task::595::TaskManager.Task::(_updateState) Task=`8a7cdef4-3793-4dd2-aafa-023c78255461`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:01:05,889::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:01:05,889::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:01:05,889::task::990::TaskManager.Task::(_decref) Task=`8a7cdef4-3793-4dd2-aafa-023c78255461`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:01:05,893::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:01:05,893::task::595::TaskManager.Task::(_updateState) Task=`2dd8d2f7-deca-430f-934d-a50d4768235d`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:01:05,894::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:01:05,894::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5df74d06-7fb2-4d43-8cc2-569ec98971bc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:01:05,894::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:01:05,895::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:01:05,895::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5df74d06-7fb2-4d43-8cc2-569ec98971bc`::Granted request >Thread-338::DEBUG::2014-05-13 12:01:05,895::task::827::TaskManager.Task::(resourceAcquired) Task=`2dd8d2f7-deca-430f-934d-a50d4768235d`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:01:05,895::task::990::TaskManager.Task::(_decref) Task=`2dd8d2f7-deca-430f-934d-a50d4768235d`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:01:05,896::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:01:05,896::task::1185::TaskManager.Task::(prepare) Task=`2dd8d2f7-deca-430f-934d-a50d4768235d`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:01:05,897::task::595::TaskManager.Task::(_updateState) Task=`2dd8d2f7-deca-430f-934d-a50d4768235d`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:01:05,897::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:01:05,897::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:01:05,897::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:01:05,897::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:01:05,897::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:01:05,898::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:01:05,898::task::990::TaskManager.Task::(_decref) Task=`2dd8d2f7-deca-430f-934d-a50d4768235d`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:01:06,238::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:06,317::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364801 s, 28.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:01:07,898::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:01:08,330::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:08,405::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031589 s, 32.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:01:10,418::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:10,504::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0424584 s, 24.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:01:10,925::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:01:12,517::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:12,599::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0370559 s, 27.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:01:13,455::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:01:13,464::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00175823 s, 2.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:01:13,952::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:01:14,612::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:14,695::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0366817 s, 27.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:01:15,916::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:01:15,917::task::595::TaskManager.Task::(_updateState) Task=`0b439dc6-dda6-495c-8114-c6e3e392fc14`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:01:15,917::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:01:15,918::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:01:15,918::task::1185::TaskManager.Task::(prepare) Task=`0b439dc6-dda6-495c-8114-c6e3e392fc14`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:01:15,918::task::595::TaskManager.Task::(_updateState) Task=`0b439dc6-dda6-495c-8114-c6e3e392fc14`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:01:15,918::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:01:15,918::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:01:15,918::task::990::TaskManager.Task::(_decref) Task=`0b439dc6-dda6-495c-8114-c6e3e392fc14`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:01:15,922::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:01:15,923::task::595::TaskManager.Task::(_updateState) Task=`940fdb8d-86e4-458e-9b5c-8bfaf6ebf3b3`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:01:15,923::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:01:15,923::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`44cd267d-0c67-4452-a449-9a2844b7039b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:01:15,923::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:01:15,924::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:01:15,924::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`44cd267d-0c67-4452-a449-9a2844b7039b`::Granted request >Thread-338::DEBUG::2014-05-13 12:01:15,924::task::827::TaskManager.Task::(resourceAcquired) Task=`940fdb8d-86e4-458e-9b5c-8bfaf6ebf3b3`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:01:15,924::task::990::TaskManager.Task::(_decref) Task=`940fdb8d-86e4-458e-9b5c-8bfaf6ebf3b3`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:01:15,925::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:01:15,925::task::1185::TaskManager.Task::(prepare) Task=`940fdb8d-86e4-458e-9b5c-8bfaf6ebf3b3`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:01:15,925::task::595::TaskManager.Task::(_updateState) Task=`940fdb8d-86e4-458e-9b5c-8bfaf6ebf3b3`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:01:15,926::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:01:15,926::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:01:15,926::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:01:15,926::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:01:15,926::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:01:15,926::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:01:15,927::task::990::TaskManager.Task::(_decref) Task=`940fdb8d-86e4-458e-9b5c-8bfaf6ebf3b3`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:01:16,708::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:16,782::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0328557 s, 31.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:01:16,970::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:01:17,115::task::595::TaskManager.Task::(_updateState) Task=`27bbfedc-d3f6-40ed-bb26-4f8f725270ce`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:01:17,115::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:01:17,116::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00175823', 'lastCheck': '3.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:01:17,116::task::1185::TaskManager.Task::(prepare) Task=`27bbfedc-d3f6-40ed-bb26-4f8f725270ce`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00175823', 'lastCheck': '3.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:01:17,116::task::595::TaskManager.Task::(_updateState) Task=`27bbfedc-d3f6-40ed-bb26-4f8f725270ce`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:01:17,116::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:01:17,116::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:01:17,116::task::990::TaskManager.Task::(_decref) Task=`27bbfedc-d3f6-40ed-bb26-4f8f725270ce`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:01:18,795::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:18,871::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0340899 s, 30.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:01:19,997::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:01:20,885::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:20,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359042 s, 28.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:01:22,977::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:01:23,017::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:01:23,057::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358162 s, 28.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:01:23,471::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:01:23,479::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000701065 s, 5.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:01:25,069::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:25,152::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0406259 s, 25.2 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:01:25,944::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:01:25,945::task::595::TaskManager.Task::(_updateState) Task=`48b06541-c618-4f53-b847-cf93963f413e`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:01:25,945::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:01:25,945::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:01:25,945::task::1185::TaskManager.Task::(prepare) Task=`48b06541-c618-4f53-b847-cf93963f413e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:01:25,946::task::595::TaskManager.Task::(_updateState) Task=`48b06541-c618-4f53-b847-cf93963f413e`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:01:25,946::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:01:25,946::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:01:25,946::task::990::TaskManager.Task::(_decref) Task=`48b06541-c618-4f53-b847-cf93963f413e`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:01:25,950::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:01:25,951::task::595::TaskManager.Task::(_updateState) Task=`34a52c46-66dc-47ed-ba3e-06e501ac8818`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:01:25,951::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:01:25,951::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`651086cd-5c12-4d4d-99e4-824701e8ff90`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:01:25,952::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:01:25,952::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:01:25,952::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`651086cd-5c12-4d4d-99e4-824701e8ff90`::Granted request >Thread-338::DEBUG::2014-05-13 12:01:25,952::task::827::TaskManager.Task::(resourceAcquired) Task=`34a52c46-66dc-47ed-ba3e-06e501ac8818`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:01:25,952::task::990::TaskManager.Task::(_decref) Task=`34a52c46-66dc-47ed-ba3e-06e501ac8818`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:01:25,953::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:01:25,953::task::1185::TaskManager.Task::(prepare) Task=`34a52c46-66dc-47ed-ba3e-06e501ac8818`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:01:25,954::task::595::TaskManager.Task::(_updateState) Task=`34a52c46-66dc-47ed-ba3e-06e501ac8818`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:01:25,954::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:01:25,954::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:01:25,954::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:01:25,954::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:01:25,955::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:01:25,955::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:01:25,955::task::990::TaskManager.Task::(_decref) Task=`34a52c46-66dc-47ed-ba3e-06e501ac8818`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:01:26,036::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:01:27,166::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:27,249::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383548 s, 26.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:01:29,057::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:01:29,262::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:29,335::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0301693 s, 33.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:01:31,348::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:31,428::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361354 s, 28.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:01:32,081::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:01:32,253::task::595::TaskManager.Task::(_updateState) Task=`a6f1a398-0a77-4d23-9508-82327cb86f4d`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:01:32,253::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:01:32,254::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000701065', 'lastCheck': '8.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:01:32,254::task::1185::TaskManager.Task::(prepare) Task=`a6f1a398-0a77-4d23-9508-82327cb86f4d`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000701065', 'lastCheck': '8.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:01:32,254::task::595::TaskManager.Task::(_updateState) Task=`a6f1a398-0a77-4d23-9508-82327cb86f4d`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:01:32,254::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:01:32,254::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:01:32,254::task::990::TaskManager.Task::(_decref) Task=`a6f1a398-0a77-4d23-9508-82327cb86f4d`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:01:33,441::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 12:01:33,508::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:01:33,526::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000601819 s, 6.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:01:33,582::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357114 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:01:35,103::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:01:35,596::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:35,670::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306195 s, 33.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:01:35,973::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:01:35,974::task::595::TaskManager.Task::(_updateState) Task=`bf9667c5-fec8-4f4c-9dcd-636496a3351d`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:01:35,974::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:01:35,974::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:01:35,974::task::1185::TaskManager.Task::(prepare) Task=`bf9667c5-fec8-4f4c-9dcd-636496a3351d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:01:35,975::task::595::TaskManager.Task::(_updateState) Task=`bf9667c5-fec8-4f4c-9dcd-636496a3351d`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:01:35,975::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:01:35,975::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:01:35,975::task::990::TaskManager.Task::(_decref) Task=`bf9667c5-fec8-4f4c-9dcd-636496a3351d`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:01:35,980::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:01:35,980::task::595::TaskManager.Task::(_updateState) Task=`a9523b68-68c6-4587-8c01-59219bed7ef2`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:01:35,980::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:01:35,981::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`93a59ac0-5817-46ee-a449-ef9da6f7dda7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:01:35,981::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:01:35,981::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:01:35,982::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`93a59ac0-5817-46ee-a449-ef9da6f7dda7`::Granted request >Thread-338::DEBUG::2014-05-13 12:01:35,982::task::827::TaskManager.Task::(resourceAcquired) Task=`a9523b68-68c6-4587-8c01-59219bed7ef2`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:01:35,982::task::990::TaskManager.Task::(_decref) Task=`a9523b68-68c6-4587-8c01-59219bed7ef2`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:01:35,983::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:01:35,983::task::1185::TaskManager.Task::(prepare) Task=`a9523b68-68c6-4587-8c01-59219bed7ef2`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:01:35,983::task::595::TaskManager.Task::(_updateState) Task=`a9523b68-68c6-4587-8c01-59219bed7ef2`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:01:35,983::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:01:35,984::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:01:35,984::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:01:35,984::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:01:35,984::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:01:35,984::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:01:35,985::task::990::TaskManager.Task::(_decref) Task=`a9523b68-68c6-4587-8c01-59219bed7ef2`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:01:37,683::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:38,083::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.358633 s, 2.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:01:38,122::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:01:40,096::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:40,171::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0328267 s, 31.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:01:41,141::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:01:42,184::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:42,263::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373909 s, 27.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:01:43,552::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:01:43,560::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000699359 s, 5.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:01:44,170::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:01:44,276::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:44,356::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373237 s, 27.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:01:46,004::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:01:46,005::task::595::TaskManager.Task::(_updateState) Task=`83ebeb2d-07f7-4d04-bbe4-759abc638b2f`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:01:46,005::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:01:46,006::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:01:46,006::task::1185::TaskManager.Task::(prepare) Task=`83ebeb2d-07f7-4d04-bbe4-759abc638b2f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:01:46,006::task::595::TaskManager.Task::(_updateState) Task=`83ebeb2d-07f7-4d04-bbe4-759abc638b2f`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:01:46,006::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:01:46,006::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:01:46,006::task::990::TaskManager.Task::(_decref) Task=`83ebeb2d-07f7-4d04-bbe4-759abc638b2f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:01:46,047::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:01:46,047::task::595::TaskManager.Task::(_updateState) Task=`b9c19991-b76c-44c3-912c-4be532817898`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:01:46,048::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:01:46,048::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a680f9ee-97e4-45de-8258-c50f455da791`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:01:46,048::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:01:46,049::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:01:46,049::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a680f9ee-97e4-45de-8258-c50f455da791`::Granted request >Thread-338::DEBUG::2014-05-13 12:01:46,049::task::827::TaskManager.Task::(resourceAcquired) Task=`b9c19991-b76c-44c3-912c-4be532817898`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:01:46,049::task::990::TaskManager.Task::(_decref) Task=`b9c19991-b76c-44c3-912c-4be532817898`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:01:46,050::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:01:46,050::task::1185::TaskManager.Task::(prepare) Task=`b9c19991-b76c-44c3-912c-4be532817898`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:01:46,050::task::595::TaskManager.Task::(_updateState) Task=`b9c19991-b76c-44c3-912c-4be532817898`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:01:46,051::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:01:46,051::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:01:46,051::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:01:46,051::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:01:46,051::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:01:46,051::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:01:46,052::task::990::TaskManager.Task::(_decref) Task=`b9c19991-b76c-44c3-912c-4be532817898`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:01:46,370::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:46,450::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0378845 s, 27.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:01:47,188::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:01:47,387::task::595::TaskManager.Task::(_updateState) Task=`2e9bab13-37b2-4e0b-b5cc-684e95228967`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:01:47,387::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:01:47,387::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000699359', 'lastCheck': '3.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:01:47,387::task::1185::TaskManager.Task::(prepare) Task=`2e9bab13-37b2-4e0b-b5cc-684e95228967`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000699359', 'lastCheck': '3.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:01:47,387::task::595::TaskManager.Task::(_updateState) Task=`2e9bab13-37b2-4e0b-b5cc-684e95228967`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:01:47,388::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:01:47,388::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:01:47,388::task::990::TaskManager.Task::(_decref) Task=`2e9bab13-37b2-4e0b-b5cc-684e95228967`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:01:48,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:48,544::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387942 s, 26.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:01:50,209::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:01:50,557::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:50,632::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0327428 s, 31.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:01:52,645::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:52,732::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0427675 s, 23.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:01:53,227::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:01:53,567::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:01:53,575::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000629694 s, 6.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:01:54,745::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:54,824::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0380749 s, 26.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:01:56,072::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:01:56,073::task::595::TaskManager.Task::(_updateState) Task=`9c0b295b-7a2f-4093-8613-029a29b40db3`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:01:56,073::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:01:56,073::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:01:56,074::task::1185::TaskManager.Task::(prepare) Task=`9c0b295b-7a2f-4093-8613-029a29b40db3`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:01:56,074::task::595::TaskManager.Task::(_updateState) Task=`9c0b295b-7a2f-4093-8613-029a29b40db3`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:01:56,074::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:01:56,074::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:01:56,074::task::990::TaskManager.Task::(_decref) Task=`9c0b295b-7a2f-4093-8613-029a29b40db3`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:01:56,079::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:01:56,079::task::595::TaskManager.Task::(_updateState) Task=`fe49e5ab-43b8-4d75-b047-600da3786e61`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:01:56,079::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:01:56,080::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`42224aed-7128-4376-9fcc-6253b1983c3d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:01:56,080::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:01:56,080::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:01:56,080::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`42224aed-7128-4376-9fcc-6253b1983c3d`::Granted request >Thread-338::DEBUG::2014-05-13 12:01:56,081::task::827::TaskManager.Task::(resourceAcquired) Task=`fe49e5ab-43b8-4d75-b047-600da3786e61`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:01:56,081::task::990::TaskManager.Task::(_decref) Task=`fe49e5ab-43b8-4d75-b047-600da3786e61`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:01:56,082::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:01:56,082::task::1185::TaskManager.Task::(prepare) Task=`fe49e5ab-43b8-4d75-b047-600da3786e61`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:01:56,082::task::595::TaskManager.Task::(_updateState) Task=`fe49e5ab-43b8-4d75-b047-600da3786e61`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:01:56,082::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:01:56,083::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:01:56,083::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:01:56,083::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:01:56,083::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:01:56,083::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:01:56,084::task::990::TaskManager.Task::(_decref) Task=`fe49e5ab-43b8-4d75-b047-600da3786e61`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:01:56,244::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:01:56,832::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:56,919::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0444744 s, 23.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:01:58,932::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:01:59,007::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322421 s, 31.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:01:59,262::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:02:01,020::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:02:01,100::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0376825 s, 27.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:02:02,281::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:02:02,513::task::595::TaskManager.Task::(_updateState) Task=`7f355258-daa8-40f5-b175-1610650f7317`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:02:02,513::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:02:02,514::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000629694', 'lastCheck': '8.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:02:02,514::task::1185::TaskManager.Task::(prepare) Task=`7f355258-daa8-40f5-b175-1610650f7317`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000629694', 'lastCheck': '8.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:02:02,514::task::595::TaskManager.Task::(_updateState) Task=`7f355258-daa8-40f5-b175-1610650f7317`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:02:02,514::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:02:02,514::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:02:02,515::task::990::TaskManager.Task::(_decref) Task=`7f355258-daa8-40f5-b175-1610650f7317`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:02:03,113::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:02:03,194::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373043 s, 27.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:02:03,583::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:02:03,590::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000697368 s, 5.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:02:05,207::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:02:05,280::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305001 s, 33.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:02:05,309::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:02:06,101::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:02:06,101::task::595::TaskManager.Task::(_updateState) Task=`ec2ab581-d598-4877-8737-5b685518a1c9`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:02:06,102::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:02:06,102::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:02:06,102::task::1185::TaskManager.Task::(prepare) Task=`ec2ab581-d598-4877-8737-5b685518a1c9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:02:06,102::task::595::TaskManager.Task::(_updateState) Task=`ec2ab581-d598-4877-8737-5b685518a1c9`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:02:06,102::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:02:06,103::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:02:06,103::task::990::TaskManager.Task::(_decref) Task=`ec2ab581-d598-4877-8737-5b685518a1c9`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:02:06,107::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:02:06,107::task::595::TaskManager.Task::(_updateState) Task=`103eee56-18b8-4106-8770-67539f86e59f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:02:06,108::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:02:06,108::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`7734e42a-c8af-4276-ae4b-eae8f6d37fe3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:02:06,108::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:02:06,109::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:02:06,109::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`7734e42a-c8af-4276-ae4b-eae8f6d37fe3`::Granted request >Thread-338::DEBUG::2014-05-13 12:02:06,109::task::827::TaskManager.Task::(resourceAcquired) Task=`103eee56-18b8-4106-8770-67539f86e59f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:02:06,109::task::990::TaskManager.Task::(_decref) Task=`103eee56-18b8-4106-8770-67539f86e59f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:02:06,110::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:02:06,110::task::1185::TaskManager.Task::(prepare) Task=`103eee56-18b8-4106-8770-67539f86e59f`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:02:06,111::task::595::TaskManager.Task::(_updateState) Task=`103eee56-18b8-4106-8770-67539f86e59f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:02:06,111::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:02:06,111::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:02:06,111::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:02:06,111::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:02:06,112::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:02:06,112::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:02:06,112::task::990::TaskManager.Task::(_decref) Task=`103eee56-18b8-4106-8770-67539f86e59f`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:02:07,293::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:02:07,379::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.042162 s, 24.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:02:08,334::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:02:09,392::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:02:09,469::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359318 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:02:11,353::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:02:11,482::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:02:11,566::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0420554 s, 24.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:02:13,579::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 12:02:13,598::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:02:13,611::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000615395 s, 6.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:02:13,686::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.064596 s, 15.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:02:14,370::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:02:15,699::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:02:15,779::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0376747 s, 27.2 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:02:16,131::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:02:16,132::task::595::TaskManager.Task::(_updateState) Task=`db099ca1-6475-49ee-83e5-fe130b50cb29`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:02:16,132::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:02:16,132::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:02:16,133::task::1185::TaskManager.Task::(prepare) Task=`db099ca1-6475-49ee-83e5-fe130b50cb29`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:02:16,133::task::595::TaskManager.Task::(_updateState) Task=`db099ca1-6475-49ee-83e5-fe130b50cb29`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:02:16,133::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:02:16,133::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:02:16,133::task::990::TaskManager.Task::(_decref) Task=`db099ca1-6475-49ee-83e5-fe130b50cb29`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:02:16,138::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:02:16,138::task::595::TaskManager.Task::(_updateState) Task=`70d19435-6078-4360-a473-9080fa84b3d3`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:02:16,139::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:02:16,139::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2f95e72c-10d3-4927-a9e3-393f15d4cdd7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:02:16,139::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:02:16,140::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:02:16,140::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2f95e72c-10d3-4927-a9e3-393f15d4cdd7`::Granted request >Thread-338::DEBUG::2014-05-13 12:02:16,140::task::827::TaskManager.Task::(resourceAcquired) Task=`70d19435-6078-4360-a473-9080fa84b3d3`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:02:16,140::task::990::TaskManager.Task::(_decref) Task=`70d19435-6078-4360-a473-9080fa84b3d3`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:02:16,141::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:02:16,141::task::1185::TaskManager.Task::(prepare) Task=`70d19435-6078-4360-a473-9080fa84b3d3`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:02:16,141::task::595::TaskManager.Task::(_updateState) Task=`70d19435-6078-4360-a473-9080fa84b3d3`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:02:16,142::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:02:16,142::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:02:16,142::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:02:16,142::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:02:16,142::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:02:16,143::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:02:16,143::task::990::TaskManager.Task::(_decref) Task=`70d19435-6078-4360-a473-9080fa84b3d3`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:02:17,392::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:02:17,645::task::595::TaskManager.Task::(_updateState) Task=`91d646cd-6546-44b0-b357-eecb51ddb7bc`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:02:17,645::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:02:17,645::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000615395', 'lastCheck': '4.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:02:17,645::task::1185::TaskManager.Task::(prepare) Task=`91d646cd-6546-44b0-b357-eecb51ddb7bc`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000615395', 'lastCheck': '4.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:02:17,646::task::595::TaskManager.Task::(_updateState) Task=`91d646cd-6546-44b0-b357-eecb51ddb7bc`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:02:17,646::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:02:17,646::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:02:17,646::task::990::TaskManager.Task::(_decref) Task=`91d646cd-6546-44b0-b357-eecb51ddb7bc`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:02:17,792::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:02:17,872::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364965 s, 28.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:02:19,885::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:02:19,958::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.030502 s, 33.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:02:20,411::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:02:21,971::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:02:22,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0423927 s, 24.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:02:23,433::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:02:23,631::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:02:23,638::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000702911 s, 5.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:02:24,069::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:02:24,162::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0505934 s, 20.2 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:02:26,160::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4478bd69] >Thread-66::DEBUG::2014-05-13 12:02:26,161::task::595::TaskManager.Task::(_updateState) Task=`131cb217-9dd5-4137-ad71-6525efa36681`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:02:26,161::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:02:26,161::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:02:26,162::task::1185::TaskManager.Task::(prepare) Task=`131cb217-9dd5-4137-ad71-6525efa36681`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:02:26,162::task::595::TaskManager.Task::(_updateState) Task=`131cb217-9dd5-4137-ad71-6525efa36681`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:02:26,162::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:02:26,162::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:02:26,162::task::990::TaskManager.Task::(_decref) Task=`131cb217-9dd5-4137-ad71-6525efa36681`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:02:26,167::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4478bd69] >Thread-338::DEBUG::2014-05-13 12:02:26,167::task::595::TaskManager.Task::(_updateState) Task=`22fb3a95-c904-47b8-ae3f-186d83bab9ac`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:02:26,167::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:02:26,168::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`25868589-0559-422d-be80-7f2e178877aa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:02:26,168::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:02:26,168::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:02:26,168::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`25868589-0559-422d-be80-7f2e178877aa`::Granted request >Thread-338::DEBUG::2014-05-13 12:02:26,169::task::827::TaskManager.Task::(resourceAcquired) Task=`22fb3a95-c904-47b8-ae3f-186d83bab9ac`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:02:26,169::task::990::TaskManager.Task::(_decref) Task=`22fb3a95-c904-47b8-ae3f-186d83bab9ac`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:02:26,169::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:02:26,170::task::1185::TaskManager.Task::(prepare) Task=`22fb3a95-c904-47b8-ae3f-186d83bab9ac`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:02:26,170::task::595::TaskManager.Task::(_updateState) Task=`22fb3a95-c904-47b8-ae3f-186d83bab9ac`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:02:26,170::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:02:26,170::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:02:26,170::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:02:26,171::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:02:26,171::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:02:26,171::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:02:26,171::task::990::TaskManager.Task::(_decref) Task=`22fb3a95-c904-47b8-ae3f-186d83bab9ac`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:02:26,175::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:02:26,253::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0351834 s, 29.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:02:26,451::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:02:28,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:02:28,341::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.033054 s, 31.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:02:29,469::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:02:30,354::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:02:30,433::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0354583 s, 28.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:02:32,446::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:02:32,496::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:02:32,531::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0390523 s, 26.2 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:02:32,776::task::595::TaskManager.Task::(_updateState) Task=`f3b9d528-1249-47ee-94e0-38beb1ee5cdc`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:02:32,776::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:02:32,777::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000702911', 'lastCheck': '9.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:02:32,777::task::1185::TaskManager.Task::(prepare) Task=`f3b9d528-1249-47ee-94e0-38beb1ee5cdc`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000702911', 'lastCheck': '9.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:02:32,777::task::595::TaskManager.Task::(_updateState) Task=`f3b9d528-1249-47ee-94e0-38beb1ee5cdc`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:02:32,777::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:02:32,777::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:02:32,778::task::990::TaskManager.Task::(_decref) Task=`f3b9d528-1249-47ee-94e0-38beb1ee5cdc`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 12:02:33,646::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:02:33,654::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000621409 s, 6.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:02:34,544::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:02:34,618::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.032581 s, 31.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:02:35,519::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:02:36,190::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:02:36,190::task::595::TaskManager.Task::(_updateState) Task=`a0156627-f472-4d13-8475-d05d22bda36b`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:02:36,191::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:02:36,191::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:02:36,191::task::1185::TaskManager.Task::(prepare) Task=`a0156627-f472-4d13-8475-d05d22bda36b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:02:36,191::task::595::TaskManager.Task::(_updateState) Task=`a0156627-f472-4d13-8475-d05d22bda36b`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:02:36,191::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:02:36,192::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:02:36,192::task::990::TaskManager.Task::(_decref) Task=`a0156627-f472-4d13-8475-d05d22bda36b`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:02:36,196::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:02:36,196::task::595::TaskManager.Task::(_updateState) Task=`6707dd0a-06b6-42f3-9bfb-7025771c320f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:02:36,197::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:02:36,198::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8b911fa3-1f8a-40e1-b9a9-dc318b82162c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:02:36,198::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:02:36,199::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:02:36,199::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8b911fa3-1f8a-40e1-b9a9-dc318b82162c`::Granted request >Thread-338::DEBUG::2014-05-13 12:02:36,199::task::827::TaskManager.Task::(resourceAcquired) Task=`6707dd0a-06b6-42f3-9bfb-7025771c320f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:02:36,199::task::990::TaskManager.Task::(_decref) Task=`6707dd0a-06b6-42f3-9bfb-7025771c320f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:02:36,200::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:02:36,200::task::1185::TaskManager.Task::(prepare) Task=`6707dd0a-06b6-42f3-9bfb-7025771c320f`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:02:36,201::task::595::TaskManager.Task::(_updateState) Task=`6707dd0a-06b6-42f3-9bfb-7025771c320f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:02:36,201::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:02:36,201::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:02:36,201::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:02:36,201::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:02:36,202::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:02:36,202::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:02:36,202::task::990::TaskManager.Task::(_decref) Task=`6707dd0a-06b6-42f3-9bfb-7025771c320f`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:02:36,631::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:02:37,002::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.328157 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:02:38,539::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:02:39,015::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:02:39,094::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035129 s, 29.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:02:41,106::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:02:41,186::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0368337 s, 27.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:02:41,557::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:02:43,198::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:02:43,273::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0326831 s, 31.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:02:43,661::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:02:43,669::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000634151 s, 6.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:02:44,586::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:02:45,286::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:02:45,365::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353226 s, 29.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:02:46,220::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:02:46,256::task::595::TaskManager.Task::(_updateState) Task=`5f6afee5-f255-46eb-b463-bdbe94bfaf4a`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:02:46,256::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:02:46,257::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:02:46,257::task::1185::TaskManager.Task::(prepare) Task=`5f6afee5-f255-46eb-b463-bdbe94bfaf4a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:02:46,257::task::595::TaskManager.Task::(_updateState) Task=`5f6afee5-f255-46eb-b463-bdbe94bfaf4a`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:02:46,257::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:02:46,257::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:02:46,258::task::990::TaskManager.Task::(_decref) Task=`5f6afee5-f255-46eb-b463-bdbe94bfaf4a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:02:46,575::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:02:46,575::task::595::TaskManager.Task::(_updateState) Task=`4ec6b56d-1143-4dd9-aca6-e5eda91e6548`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:02:46,575::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:02:46,576::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`614a459f-32d6-4546-b37d-a11d1bb039fd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:02:46,576::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:02:46,576::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:02:46,576::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`614a459f-32d6-4546-b37d-a11d1bb039fd`::Granted request >Thread-338::DEBUG::2014-05-13 12:02:46,577::task::827::TaskManager.Task::(resourceAcquired) Task=`4ec6b56d-1143-4dd9-aca6-e5eda91e6548`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:02:46,577::task::990::TaskManager.Task::(_decref) Task=`4ec6b56d-1143-4dd9-aca6-e5eda91e6548`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:02:46,578::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:02:46,578::task::1185::TaskManager.Task::(prepare) Task=`4ec6b56d-1143-4dd9-aca6-e5eda91e6548`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:02:46,578::task::595::TaskManager.Task::(_updateState) Task=`4ec6b56d-1143-4dd9-aca6-e5eda91e6548`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:02:46,578::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:02:46,578::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:02:46,579::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:02:46,579::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:02:46,579::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:02:46,579::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:02:46,579::task::990::TaskManager.Task::(_decref) Task=`4ec6b56d-1143-4dd9-aca6-e5eda91e6548`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:02:47,378::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:02:47,457::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0370855 s, 27.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:02:47,604::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:02:47,907::task::595::TaskManager.Task::(_updateState) Task=`f5f55b57-fe12-4830-a0b3-4ef20056ae06`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:02:47,908::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:02:47,908::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000634151', 'lastCheck': '4.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:02:47,908::task::1185::TaskManager.Task::(prepare) Task=`f5f55b57-fe12-4830-a0b3-4ef20056ae06`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000634151', 'lastCheck': '4.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:02:47,908::task::595::TaskManager.Task::(_updateState) Task=`f5f55b57-fe12-4830-a0b3-4ef20056ae06`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:02:47,908::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:02:47,909::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:02:47,909::task::990::TaskManager.Task::(_decref) Task=`f5f55b57-fe12-4830-a0b3-4ef20056ae06`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:02:49,470::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:02:49,543::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0302936 s, 33.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:02:50,621::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:02:51,556::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:02:51,922::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.324922 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:02:53,641::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:02:53,676::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:02:53,684::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000611959 s, 6.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:02:53,935::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:02:54,018::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359683 s, 28.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:02:56,031::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:02:56,105::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313051 s, 32.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:02:56,601::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3e537ec9] >Thread-66::DEBUG::2014-05-13 12:02:56,614::task::595::TaskManager.Task::(_updateState) Task=`a4e625fa-cd77-4f8c-852c-59c0320f1357`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:02:56,615::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:02:56,615::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:02:56,615::task::1185::TaskManager.Task::(prepare) Task=`a4e625fa-cd77-4f8c-852c-59c0320f1357`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:02:56,615::task::595::TaskManager.Task::(_updateState) Task=`a4e625fa-cd77-4f8c-852c-59c0320f1357`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:02:56,616::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:02:56,616::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:02:56,616::task::990::TaskManager.Task::(_decref) Task=`a4e625fa-cd77-4f8c-852c-59c0320f1357`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:02:56,621::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3e537ec9] >Thread-338::DEBUG::2014-05-13 12:02:56,621::task::595::TaskManager.Task::(_updateState) Task=`d38e3d09-4288-44ff-90ad-4a7b7e3f8991`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:02:56,621::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:02:56,622::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`43d4df91-f659-458d-8f5e-41fc16450403`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:02:56,622::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:02:56,622::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:02:56,622::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`43d4df91-f659-458d-8f5e-41fc16450403`::Granted request >Thread-338::DEBUG::2014-05-13 12:02:56,623::task::827::TaskManager.Task::(resourceAcquired) Task=`d38e3d09-4288-44ff-90ad-4a7b7e3f8991`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:02:56,623::task::990::TaskManager.Task::(_decref) Task=`d38e3d09-4288-44ff-90ad-4a7b7e3f8991`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:02:56,624::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:02:56,624::task::1185::TaskManager.Task::(prepare) Task=`d38e3d09-4288-44ff-90ad-4a7b7e3f8991`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:02:56,624::task::595::TaskManager.Task::(_updateState) Task=`d38e3d09-4288-44ff-90ad-4a7b7e3f8991`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:02:56,624::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:02:56,625::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:02:56,625::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:02:56,625::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:02:56,625::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:02:56,625::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:02:56,626::task::990::TaskManager.Task::(_decref) Task=`d38e3d09-4288-44ff-90ad-4a7b7e3f8991`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:02:56,659::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:02:58,117::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:02:58,191::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313378 s, 32.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:02:59,682::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:03:00,204::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:00,292::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0458957 s, 22.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:03:02,305::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:02,384::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365943 s, 28.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:03:02,702::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:03:03,040::task::595::TaskManager.Task::(_updateState) Task=`e6586d3a-aea1-4a30-bb71-05fbae603b8e`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:03:03,040::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:03:03,040::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000611959', 'lastCheck': '9.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:03:03,041::task::1185::TaskManager.Task::(prepare) Task=`e6586d3a-aea1-4a30-bb71-05fbae603b8e`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000611959', 'lastCheck': '9.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:03:03,041::task::595::TaskManager.Task::(_updateState) Task=`e6586d3a-aea1-4a30-bb71-05fbae603b8e`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:03:03,041::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:03:03,041::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:03:03,041::task::990::TaskManager.Task::(_decref) Task=`e6586d3a-aea1-4a30-bb71-05fbae603b8e`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 12:03:03,691::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:03:03,700::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00161536 s, 2.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:03:04,397::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:04,472::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0339958 s, 30.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:03:05,719::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:03:06,485::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:06,577::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0501109 s, 20.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:03:06,644::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:03:06,644::task::595::TaskManager.Task::(_updateState) Task=`9eb6527e-d61b-44f2-abd7-830ce58e70c7`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:03:06,645::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:03:06,645::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:03:06,645::task::1185::TaskManager.Task::(prepare) Task=`9eb6527e-d61b-44f2-abd7-830ce58e70c7`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:03:06,645::task::595::TaskManager.Task::(_updateState) Task=`9eb6527e-d61b-44f2-abd7-830ce58e70c7`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:03:06,645::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:03:06,646::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:03:06,646::task::990::TaskManager.Task::(_decref) Task=`9eb6527e-d61b-44f2-abd7-830ce58e70c7`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:03:06,650::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:03:06,651::task::595::TaskManager.Task::(_updateState) Task=`a88c3e78-680d-4b22-a5c0-a3694f5ed8f4`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:03:06,651::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:03:06,651::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a7fc6f47-f2a2-4b4c-906d-8a6b920bcfec`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:03:06,652::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:03:06,652::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:03:06,652::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a7fc6f47-f2a2-4b4c-906d-8a6b920bcfec`::Granted request >Thread-338::DEBUG::2014-05-13 12:03:06,652::task::827::TaskManager.Task::(resourceAcquired) Task=`a88c3e78-680d-4b22-a5c0-a3694f5ed8f4`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:03:06,653::task::990::TaskManager.Task::(_decref) Task=`a88c3e78-680d-4b22-a5c0-a3694f5ed8f4`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:03:06,653::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:03:06,653::task::1185::TaskManager.Task::(prepare) Task=`a88c3e78-680d-4b22-a5c0-a3694f5ed8f4`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:03:06,654::task::595::TaskManager.Task::(_updateState) Task=`a88c3e78-680d-4b22-a5c0-a3694f5ed8f4`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:03:06,654::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:03:06,654::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:03:06,654::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:03:06,654::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:03:06,655::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:03:06,655::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:03:06,655::task::990::TaskManager.Task::(_decref) Task=`a88c3e78-680d-4b22-a5c0-a3694f5ed8f4`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:03:08,591::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:08,666::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0331028 s, 30.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:03:08,738::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:03:10,679::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:10,754::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0330666 s, 31.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:03:11,760::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:03:12,767::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:12,847::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0351703 s, 29.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:03:13,707::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:03:13,716::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00166058 s, 2.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:03:14,783::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:03:14,861::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:14,938::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035429 s, 28.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:03:16,673::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:03:16,673::task::595::TaskManager.Task::(_updateState) Task=`7ca55a52-1b9d-40ba-8a02-c34696a71663`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:03:16,673::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:03:16,674::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:03:16,674::task::1185::TaskManager.Task::(prepare) Task=`7ca55a52-1b9d-40ba-8a02-c34696a71663`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:03:16,674::task::595::TaskManager.Task::(_updateState) Task=`7ca55a52-1b9d-40ba-8a02-c34696a71663`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:03:16,674::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:03:16,675::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:03:16,675::task::990::TaskManager.Task::(_decref) Task=`7ca55a52-1b9d-40ba-8a02-c34696a71663`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:03:16,679::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:03:16,679::task::595::TaskManager.Task::(_updateState) Task=`57cd9855-f1da-4151-a107-a6c232416dbb`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:03:16,680::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:03:16,680::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`258b920d-d434-44a1-bfaf-743c1ad26c6f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:03:16,680::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:03:16,681::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:03:16,681::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`258b920d-d434-44a1-bfaf-743c1ad26c6f`::Granted request >Thread-338::DEBUG::2014-05-13 12:03:16,681::task::827::TaskManager.Task::(resourceAcquired) Task=`57cd9855-f1da-4151-a107-a6c232416dbb`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:03:16,681::task::990::TaskManager.Task::(_decref) Task=`57cd9855-f1da-4151-a107-a6c232416dbb`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:03:16,682::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:03:16,682::task::1185::TaskManager.Task::(prepare) Task=`57cd9855-f1da-4151-a107-a6c232416dbb`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:03:16,682::task::595::TaskManager.Task::(_updateState) Task=`57cd9855-f1da-4151-a107-a6c232416dbb`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:03:16,683::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:03:16,683::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:03:16,683::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:03:16,683::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:03:16,683::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:03:16,684::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:03:16,684::task::990::TaskManager.Task::(_decref) Task=`57cd9855-f1da-4151-a107-a6c232416dbb`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:03:16,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:17,029::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352543 s, 29.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:03:17,801::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:03:18,171::task::595::TaskManager.Task::(_updateState) Task=`bef66ef9-c3b9-44a9-bb82-57dec2269656`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:03:18,171::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:03:18,172::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00166058', 'lastCheck': '4.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:03:18,172::task::1185::TaskManager.Task::(prepare) Task=`bef66ef9-c3b9-44a9-bb82-57dec2269656`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00166058', 'lastCheck': '4.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:03:18,172::task::595::TaskManager.Task::(_updateState) Task=`bef66ef9-c3b9-44a9-bb82-57dec2269656`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:03:18,172::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:03:18,172::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:03:18,173::task::990::TaskManager.Task::(_decref) Task=`bef66ef9-c3b9-44a9-bb82-57dec2269656`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:03:19,042::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:19,127::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0419002 s, 24.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:03:20,819::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:03:21,141::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:21,221::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0368955 s, 27.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:03:23,234::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:23,307::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0304338 s, 33.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:03:23,747::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:03:23,755::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000703638 s, 5.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:03:23,838::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:03:25,320::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:25,393::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303867 s, 33.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:03:26,707::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1cb1b71a] >Thread-66::DEBUG::2014-05-13 12:03:26,708::task::595::TaskManager.Task::(_updateState) Task=`175038b2-d224-425d-821e-5ec0e53da713`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:03:26,708::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:03:26,709::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:03:26,709::task::1185::TaskManager.Task::(prepare) Task=`175038b2-d224-425d-821e-5ec0e53da713`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:03:26,709::task::595::TaskManager.Task::(_updateState) Task=`175038b2-d224-425d-821e-5ec0e53da713`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:03:26,709::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:03:26,709::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:03:26,709::task::990::TaskManager.Task::(_decref) Task=`175038b2-d224-425d-821e-5ec0e53da713`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:03:26,714::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1cb1b71a] >Thread-338::DEBUG::2014-05-13 12:03:26,714::task::595::TaskManager.Task::(_updateState) Task=`adad457b-52b0-4829-9ab2-ba26a257a3e3`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:03:26,715::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:03:26,715::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`82100add-0fac-4e92-a163-f3007a01286a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:03:26,715::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:03:26,715::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:03:26,716::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`82100add-0fac-4e92-a163-f3007a01286a`::Granted request >Thread-338::DEBUG::2014-05-13 12:03:26,716::task::827::TaskManager.Task::(resourceAcquired) Task=`adad457b-52b0-4829-9ab2-ba26a257a3e3`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:03:26,716::task::990::TaskManager.Task::(_decref) Task=`adad457b-52b0-4829-9ab2-ba26a257a3e3`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:03:26,717::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:03:26,717::task::1185::TaskManager.Task::(prepare) Task=`adad457b-52b0-4829-9ab2-ba26a257a3e3`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:03:26,717::task::595::TaskManager.Task::(_updateState) Task=`adad457b-52b0-4829-9ab2-ba26a257a3e3`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:03:26,717::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:03:26,718::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:03:26,718::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:03:26,718::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:03:26,718::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:03:26,718::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:03:26,719::task::990::TaskManager.Task::(_decref) Task=`adad457b-52b0-4829-9ab2-ba26a257a3e3`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:03:26,856::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:03:27,407::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:27,488::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362632 s, 28.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:03:29,501::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:29,575::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0304697 s, 33.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:03:29,880::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:03:31,588::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:31,666::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353165 s, 29.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:03:32,909::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:03:33,305::task::595::TaskManager.Task::(_updateState) Task=`ad464e62-1899-4fee-9a2b-c77d1ff25e79`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:03:33,305::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:03:33,306::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000703638', 'lastCheck': '9.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:03:33,306::task::1185::TaskManager.Task::(prepare) Task=`ad464e62-1899-4fee-9a2b-c77d1ff25e79`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000703638', 'lastCheck': '9.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:03:33,306::task::595::TaskManager.Task::(_updateState) Task=`ad464e62-1899-4fee-9a2b-c77d1ff25e79`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:03:33,306::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:03:33,306::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:03:33,307::task::990::TaskManager.Task::(_decref) Task=`ad464e62-1899-4fee-9a2b-c77d1ff25e79`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:03:33,679::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 12:03:33,778::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:03:33,792::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000593082 s, 6.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:03:33,815::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0479914 s, 21.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:03:35,829::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:35,908::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360282 s, 28.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:03:35,933::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:03:36,737::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:03:36,758::task::595::TaskManager.Task::(_updateState) Task=`67701dad-dfbd-4a43-8b27-e5fa0dbeba17`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:03:36,758::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:03:36,759::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:03:36,759::task::1185::TaskManager.Task::(prepare) Task=`67701dad-dfbd-4a43-8b27-e5fa0dbeba17`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:03:36,759::task::595::TaskManager.Task::(_updateState) Task=`67701dad-dfbd-4a43-8b27-e5fa0dbeba17`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:03:36,759::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:03:36,759::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:03:36,760::task::990::TaskManager.Task::(_decref) Task=`67701dad-dfbd-4a43-8b27-e5fa0dbeba17`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:03:36,764::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:03:36,765::task::595::TaskManager.Task::(_updateState) Task=`66e1c921-1c95-44be-adf4-7279f68e39f0`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:03:36,765::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:03:36,765::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`635e8474-9e33-4167-8b17-6317eaae3dce`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:03:36,765::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:03:36,766::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:03:36,766::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`635e8474-9e33-4167-8b17-6317eaae3dce`::Granted request >Thread-338::DEBUG::2014-05-13 12:03:36,766::task::827::TaskManager.Task::(resourceAcquired) Task=`66e1c921-1c95-44be-adf4-7279f68e39f0`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:03:36,766::task::990::TaskManager.Task::(_decref) Task=`66e1c921-1c95-44be-adf4-7279f68e39f0`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:03:36,767::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:03:36,767::task::1185::TaskManager.Task::(prepare) Task=`66e1c921-1c95-44be-adf4-7279f68e39f0`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:03:36,768::task::595::TaskManager.Task::(_updateState) Task=`66e1c921-1c95-44be-adf4-7279f68e39f0`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:03:36,768::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:03:36,768::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:03:36,768::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:03:36,768::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:03:36,769::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:03:36,769::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:03:36,769::task::990::TaskManager.Task::(_decref) Task=`66e1c921-1c95-44be-adf4-7279f68e39f0`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:03:37,921::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:37,994::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307353 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:03:38,953::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:03:40,007::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:40,085::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035098 s, 29.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:03:41,973::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:03:42,098::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:42,177::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353814 s, 28.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:03:43,811::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:03:43,819::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000666418 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:03:44,190::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:44,267::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.03519 s, 29.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:03:44,992::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:03:46,280::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:46,359::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0304981 s, 33.6 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:03:46,789::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6849800f] >Thread-66::DEBUG::2014-05-13 12:03:46,789::task::595::TaskManager.Task::(_updateState) Task=`4b4a7a39-1290-47e8-affc-8b93cbf365cd`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:03:46,789::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:03:46,790::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:03:46,790::task::1185::TaskManager.Task::(prepare) Task=`4b4a7a39-1290-47e8-affc-8b93cbf365cd`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:03:46,790::task::595::TaskManager.Task::(_updateState) Task=`4b4a7a39-1290-47e8-affc-8b93cbf365cd`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:03:46,790::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:03:46,790::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:03:46,791::task::990::TaskManager.Task::(_decref) Task=`4b4a7a39-1290-47e8-affc-8b93cbf365cd`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:03:46,795::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6849800f] >Thread-338::DEBUG::2014-05-13 12:03:46,795::task::595::TaskManager.Task::(_updateState) Task=`30079d9f-cf43-4cde-8ce9-04a370468588`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:03:46,796::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:03:46,796::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9186130d-2710-48d3-91e2-7729f345592d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:03:46,796::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:03:46,797::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:03:46,797::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9186130d-2710-48d3-91e2-7729f345592d`::Granted request >Thread-338::DEBUG::2014-05-13 12:03:46,798::task::827::TaskManager.Task::(resourceAcquired) Task=`30079d9f-cf43-4cde-8ce9-04a370468588`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:03:46,798::task::990::TaskManager.Task::(_decref) Task=`30079d9f-cf43-4cde-8ce9-04a370468588`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:03:46,799::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:03:46,799::task::1185::TaskManager.Task::(prepare) Task=`30079d9f-cf43-4cde-8ce9-04a370468588`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:03:46,799::task::595::TaskManager.Task::(_updateState) Task=`30079d9f-cf43-4cde-8ce9-04a370468588`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:03:46,800::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:03:46,800::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:03:46,800::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:03:46,800::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:03:46,800::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:03:46,800::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:03:46,801::task::990::TaskManager.Task::(_decref) Task=`30079d9f-cf43-4cde-8ce9-04a370468588`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:03:48,010::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:03:48,375::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 12:03:48,438::task::595::TaskManager.Task::(_updateState) Task=`8850b0c6-7a5d-41ef-be5a-accedd0f5408`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:03:48,440::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:03:48,442::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000666418', 'lastCheck': '4.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:03:48,443::task::1185::TaskManager.Task::(prepare) Task=`8850b0c6-7a5d-41ef-be5a-accedd0f5408`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000666418', 'lastCheck': '4.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:03:48,445::task::595::TaskManager.Task::(_updateState) Task=`8850b0c6-7a5d-41ef-be5a-accedd0f5408`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:03:48,446::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:03:48,447::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:03:48,449::task::990::TaskManager.Task::(_decref) Task=`8850b0c6-7a5d-41ef-be5a-accedd0f5408`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:03:48,622::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.204305 s, 5.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:03:50,635::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:50,712::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0330258 s, 31.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:03:51,035::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:03:52,725::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:52,799::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0329464 s, 31.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:03:53,826::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:03:53,834::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000651785 s, 6.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:03:54,053::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:03:54,812::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:54,896::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387995 s, 26.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:03:56,819::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1c80df6] >Thread-66::DEBUG::2014-05-13 12:03:56,820::task::595::TaskManager.Task::(_updateState) Task=`91f0eae1-c58b-461a-8666-ac9be062802e`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:03:56,820::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:03:56,821::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:03:56,821::task::1185::TaskManager.Task::(prepare) Task=`91f0eae1-c58b-461a-8666-ac9be062802e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:03:56,821::task::595::TaskManager.Task::(_updateState) Task=`91f0eae1-c58b-461a-8666-ac9be062802e`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:03:56,821::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:03:56,821::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:03:56,821::task::990::TaskManager.Task::(_decref) Task=`91f0eae1-c58b-461a-8666-ac9be062802e`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:03:56,826::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1c80df6] >Thread-338::DEBUG::2014-05-13 12:03:56,826::task::595::TaskManager.Task::(_updateState) Task=`55f89e29-ad79-411f-a1df-ecbf0631fb60`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:03:56,826::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:03:56,827::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`83cfc2d2-eff7-48e7-bbc5-577a561c0235`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:03:56,827::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:03:56,827::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:03:56,827::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`83cfc2d2-eff7-48e7-bbc5-577a561c0235`::Granted request >Thread-338::DEBUG::2014-05-13 12:03:56,828::task::827::TaskManager.Task::(resourceAcquired) Task=`55f89e29-ad79-411f-a1df-ecbf0631fb60`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:03:56,828::task::990::TaskManager.Task::(_decref) Task=`55f89e29-ad79-411f-a1df-ecbf0631fb60`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:03:56,829::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:03:56,829::task::1185::TaskManager.Task::(prepare) Task=`55f89e29-ad79-411f-a1df-ecbf0631fb60`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:03:56,829::task::595::TaskManager.Task::(_updateState) Task=`55f89e29-ad79-411f-a1df-ecbf0631fb60`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:03:56,829::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:03:56,829::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:03:56,830::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:03:56,830::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:03:56,830::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:03:56,830::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:03:56,830::task::990::TaskManager.Task::(_decref) Task=`55f89e29-ad79-411f-a1df-ecbf0631fb60`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:03:56,909::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:56,988::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035609 s, 28.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:03:57,076::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:03:59,000::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:03:59,074::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322534 s, 31.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:04:00,095::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:04:01,088::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:01,450::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.310874 s, 3.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:04:03,116::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:04:03,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:03,546::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0388353 s, 26.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:04:03,607::task::595::TaskManager.Task::(_updateState) Task=`bb1de29f-cdbf-4a00-bc04-7d6e74f7efb3`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:04:03,607::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:04:03,607::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000651785', 'lastCheck': '9.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:04:03,607::task::1185::TaskManager.Task::(prepare) Task=`bb1de29f-cdbf-4a00-bc04-7d6e74f7efb3`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000651785', 'lastCheck': '9.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:04:03,608::task::595::TaskManager.Task::(_updateState) Task=`bb1de29f-cdbf-4a00-bc04-7d6e74f7efb3`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:04:03,608::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:04:03,608::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:04:03,608::task::990::TaskManager.Task::(_decref) Task=`bb1de29f-cdbf-4a00-bc04-7d6e74f7efb3`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 12:04:03,841::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:04:03,849::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000671553 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:04:05,559::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:05,633::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307554 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:04:06,139::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:04:06,849::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17998964] >Thread-66::DEBUG::2014-05-13 12:04:06,850::task::595::TaskManager.Task::(_updateState) Task=`7d82647f-a296-46cf-b953-971986749080`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:04:06,850::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:04:06,850::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:04:06,851::task::1185::TaskManager.Task::(prepare) Task=`7d82647f-a296-46cf-b953-971986749080`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:04:06,851::task::595::TaskManager.Task::(_updateState) Task=`7d82647f-a296-46cf-b953-971986749080`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:04:06,851::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:04:06,851::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:04:06,851::task::990::TaskManager.Task::(_decref) Task=`7d82647f-a296-46cf-b953-971986749080`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:04:06,856::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17998964] >Thread-338::DEBUG::2014-05-13 12:04:06,856::task::595::TaskManager.Task::(_updateState) Task=`ac1af30a-5572-4195-8046-bb59961a2d9f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:04:06,856::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:04:06,857::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c9844374-0aaf-4c7c-a99a-1556cc5938d6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:04:06,857::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:04:06,857::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:04:06,858::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c9844374-0aaf-4c7c-a99a-1556cc5938d6`::Granted request >Thread-338::DEBUG::2014-05-13 12:04:06,858::task::827::TaskManager.Task::(resourceAcquired) Task=`ac1af30a-5572-4195-8046-bb59961a2d9f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:04:06,858::task::990::TaskManager.Task::(_decref) Task=`ac1af30a-5572-4195-8046-bb59961a2d9f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:04:06,859::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:04:06,859::task::1185::TaskManager.Task::(prepare) Task=`ac1af30a-5572-4195-8046-bb59961a2d9f`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:04:06,859::task::595::TaskManager.Task::(_updateState) Task=`ac1af30a-5572-4195-8046-bb59961a2d9f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:04:06,859::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:04:06,860::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:04:06,860::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:04:06,860::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:04:06,860::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:04:06,860::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:04:06,861::task::990::TaskManager.Task::(_decref) Task=`ac1af30a-5572-4195-8046-bb59961a2d9f`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:04:07,646::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:07,719::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312572 s, 32.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:04:09,160::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:04:09,732::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:09,813::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363055 s, 28.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:04:11,826::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:11,904::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355927 s, 28.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:04:12,179::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:04:13,856::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:04:13,864::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000706817 s, 5.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:04:13,917::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:13,996::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037476 s, 27.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:04:15,197::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:04:16,009::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:16,096::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0438965 s, 23.3 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:04:16,882::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:04:16,913::task::595::TaskManager.Task::(_updateState) Task=`b1889de2-639f-45cb-9104-326d281b7e03`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:04:16,914::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:04:16,914::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:04:16,914::task::1185::TaskManager.Task::(prepare) Task=`b1889de2-639f-45cb-9104-326d281b7e03`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:04:16,914::task::595::TaskManager.Task::(_updateState) Task=`b1889de2-639f-45cb-9104-326d281b7e03`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:04:16,914::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:04:16,915::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:04:16,915::task::990::TaskManager.Task::(_decref) Task=`b1889de2-639f-45cb-9104-326d281b7e03`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:04:16,920::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:04:16,920::task::595::TaskManager.Task::(_updateState) Task=`59277ab7-7628-4da5-b60b-6ddd08a26cfb`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:04:16,920::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:04:16,921::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6a61cefa-aa15-4093-b87c-98c6f6e0279f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:04:16,921::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:04:16,921::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:04:16,922::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6a61cefa-aa15-4093-b87c-98c6f6e0279f`::Granted request >Thread-338::DEBUG::2014-05-13 12:04:16,922::task::827::TaskManager.Task::(resourceAcquired) Task=`59277ab7-7628-4da5-b60b-6ddd08a26cfb`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:04:16,922::task::990::TaskManager.Task::(_decref) Task=`59277ab7-7628-4da5-b60b-6ddd08a26cfb`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:04:16,923::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:04:16,923::task::1185::TaskManager.Task::(prepare) Task=`59277ab7-7628-4da5-b60b-6ddd08a26cfb`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:04:16,923::task::595::TaskManager.Task::(_updateState) Task=`59277ab7-7628-4da5-b60b-6ddd08a26cfb`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:04:16,923::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:04:16,924::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:04:16,924::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:04:16,924::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:04:16,924::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:04:16,924::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:04:16,925::task::990::TaskManager.Task::(_decref) Task=`59277ab7-7628-4da5-b60b-6ddd08a26cfb`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:04:18,110::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:18,190::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373403 s, 27.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:04:18,217::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:04:18,741::task::595::TaskManager.Task::(_updateState) Task=`f6a2636d-2705-4c33-80f2-4a829031f9c8`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:04:18,741::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:04:18,741::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000706817', 'lastCheck': '4.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:04:18,742::task::1185::TaskManager.Task::(prepare) Task=`f6a2636d-2705-4c33-80f2-4a829031f9c8`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000706817', 'lastCheck': '4.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:04:18,742::task::595::TaskManager.Task::(_updateState) Task=`f6a2636d-2705-4c33-80f2-4a829031f9c8`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:04:18,742::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:04:18,742::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:04:18,742::task::990::TaskManager.Task::(_decref) Task=`f6a2636d-2705-4c33-80f2-4a829031f9c8`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:04:20,203::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:20,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323344 s, 31.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:04:21,237::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:04:22,292::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:22,369::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0325225 s, 31.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:04:23,871::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:04:23,879::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00057838 s, 7.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:04:24,255::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:04:24,382::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:24,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382849 s, 26.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:04:26,476::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:26,566::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0438489 s, 23.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:04:26,944::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:04:26,945::task::595::TaskManager.Task::(_updateState) Task=`c6e55934-14b1-444c-9350-29938a02f7a4`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:04:26,945::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:04:26,945::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:04:26,945::task::1185::TaskManager.Task::(prepare) Task=`c6e55934-14b1-444c-9350-29938a02f7a4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:04:26,946::task::595::TaskManager.Task::(_updateState) Task=`c6e55934-14b1-444c-9350-29938a02f7a4`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:04:26,946::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:04:26,946::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:04:26,946::task::990::TaskManager.Task::(_decref) Task=`c6e55934-14b1-444c-9350-29938a02f7a4`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:04:26,951::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:04:26,951::task::595::TaskManager.Task::(_updateState) Task=`cd5ba6a2-9a65-48ef-9bbe-303913ec6f9a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:04:26,951::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:04:26,952::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`62ded838-bbf8-480f-8adb-1316f1c8491b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:04:26,952::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:04:26,952::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:04:26,952::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`62ded838-bbf8-480f-8adb-1316f1c8491b`::Granted request >Thread-338::DEBUG::2014-05-13 12:04:26,953::task::827::TaskManager.Task::(resourceAcquired) Task=`cd5ba6a2-9a65-48ef-9bbe-303913ec6f9a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:04:26,953::task::990::TaskManager.Task::(_decref) Task=`cd5ba6a2-9a65-48ef-9bbe-303913ec6f9a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:04:26,954::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:04:26,954::task::1185::TaskManager.Task::(prepare) Task=`cd5ba6a2-9a65-48ef-9bbe-303913ec6f9a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:04:26,954::task::595::TaskManager.Task::(_updateState) Task=`cd5ba6a2-9a65-48ef-9bbe-303913ec6f9a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:04:26,954::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:04:26,954::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:04:26,955::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:04:26,955::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:04:26,955::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:04:26,955::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:04:26,955::task::990::TaskManager.Task::(_decref) Task=`cd5ba6a2-9a65-48ef-9bbe-303913ec6f9a`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:04:27,274::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:04:28,579::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:28,656::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0347613 s, 29.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:04:30,296::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:04:30,669::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:30,759::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0468844 s, 21.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:04:32,772::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:32,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0380038 s, 26.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:04:33,315::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:04:33,872::task::595::TaskManager.Task::(_updateState) Task=`9809fcca-67db-4304-aede-e89733a67af6`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:04:33,872::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:04:33,872::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00057838', 'lastCheck': '10.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:04:33,872::task::1185::TaskManager.Task::(prepare) Task=`9809fcca-67db-4304-aede-e89733a67af6`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00057838', 'lastCheck': '10.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:04:33,873::task::595::TaskManager.Task::(_updateState) Task=`9809fcca-67db-4304-aede-e89733a67af6`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:04:33,873::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:04:33,873::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:04:33,873::task::990::TaskManager.Task::(_decref) Task=`9809fcca-67db-4304-aede-e89733a67af6`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 12:04:33,886::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:04:33,895::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00060946 s, 6.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:04:34,865::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:34,940::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0338638 s, 30.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:04:36,333::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:04:36,954::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 12:04:36,974::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:04:36,975::task::595::TaskManager.Task::(_updateState) Task=`5d665c20-da29-4867-b2e9-51e951a32fcb`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:04:36,977::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:04:36,978::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:04:36,979::task::1185::TaskManager.Task::(prepare) Task=`5d665c20-da29-4867-b2e9-51e951a32fcb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:04:36,980::task::595::TaskManager.Task::(_updateState) Task=`5d665c20-da29-4867-b2e9-51e951a32fcb`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:04:36,981::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:04:36,982::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:04:36,983::task::990::TaskManager.Task::(_decref) Task=`5d665c20-da29-4867-b2e9-51e951a32fcb`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:04:36,989::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:04:36,989::task::595::TaskManager.Task::(_updateState) Task=`3dcdcfe2-2efa-43b3-9881-7c790c7a8d6d`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:04:36,990::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:04:36,990::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`050976d4-54fc-44ac-886f-d2cbbd636b49`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:04:36,990::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:04:36,991::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:04:36,991::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`050976d4-54fc-44ac-886f-d2cbbd636b49`::Granted request >Thread-338::DEBUG::2014-05-13 12:04:36,992::task::827::TaskManager.Task::(resourceAcquired) Task=`3dcdcfe2-2efa-43b3-9881-7c790c7a8d6d`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:04:36,992::task::990::TaskManager.Task::(_decref) Task=`3dcdcfe2-2efa-43b3-9881-7c790c7a8d6d`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:04:36,993::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:04:36,993::task::1185::TaskManager.Task::(prepare) Task=`3dcdcfe2-2efa-43b3-9881-7c790c7a8d6d`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:04:36,993::task::595::TaskManager.Task::(_updateState) Task=`3dcdcfe2-2efa-43b3-9881-7c790c7a8d6d`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:04:36,993::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:04:36,994::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:04:36,994::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:04:36,994::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:04:36,994::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:04:36,995::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:04:36,995::task::990::TaskManager.Task::(_decref) Task=`3dcdcfe2-2efa-43b3-9881-7c790c7a8d6d`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:04:37,046::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0500123 s, 20.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:04:39,059::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:39,139::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364484 s, 28.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:04:39,356::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:04:41,152::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:41,227::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313233 s, 32.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:04:42,375::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:04:43,240::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:43,648::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.365306 s, 2.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:04:43,905::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:04:43,913::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000644802 s, 6.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:04:45,395::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:04:45,661::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:45,740::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361042 s, 28.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:04:47,014::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:04:47,014::task::595::TaskManager.Task::(_updateState) Task=`3f41a5e2-e450-4530-90ed-841d227c119b`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:04:47,015::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:04:47,015::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:04:47,015::task::1185::TaskManager.Task::(prepare) Task=`3f41a5e2-e450-4530-90ed-841d227c119b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:04:47,015::task::595::TaskManager.Task::(_updateState) Task=`3f41a5e2-e450-4530-90ed-841d227c119b`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:04:47,015::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:04:47,016::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:04:47,016::task::990::TaskManager.Task::(_decref) Task=`3f41a5e2-e450-4530-90ed-841d227c119b`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:04:47,020::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:04:47,021::task::595::TaskManager.Task::(_updateState) Task=`047f41da-d26e-436c-9591-545d91055793`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:04:47,021::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:04:47,021::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`81895a1d-256b-41dc-8475-ff36c5e35393`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:04:47,021::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:04:47,022::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:04:47,022::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`81895a1d-256b-41dc-8475-ff36c5e35393`::Granted request >Thread-338::DEBUG::2014-05-13 12:04:47,022::task::827::TaskManager.Task::(resourceAcquired) Task=`047f41da-d26e-436c-9591-545d91055793`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:04:47,022::task::990::TaskManager.Task::(_decref) Task=`047f41da-d26e-436c-9591-545d91055793`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:04:47,023::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:04:47,023::task::1185::TaskManager.Task::(prepare) Task=`047f41da-d26e-436c-9591-545d91055793`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:04:47,024::task::595::TaskManager.Task::(_updateState) Task=`047f41da-d26e-436c-9591-545d91055793`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:04:47,024::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:04:47,024::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:04:47,024::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:04:47,024::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:04:47,025::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:04:47,025::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:04:47,025::task::990::TaskManager.Task::(_decref) Task=`047f41da-d26e-436c-9591-545d91055793`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:04:47,753::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:47,835::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0386445 s, 26.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:04:48,426::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:04:49,021::task::595::TaskManager.Task::(_updateState) Task=`6ff0c07c-be66-4756-8723-e7be044bc660`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:04:49,022::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:04:49,022::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000644802', 'lastCheck': '5.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:04:49,022::task::1185::TaskManager.Task::(prepare) Task=`6ff0c07c-be66-4756-8723-e7be044bc660`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000644802', 'lastCheck': '5.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:04:49,022::task::595::TaskManager.Task::(_updateState) Task=`6ff0c07c-be66-4756-8723-e7be044bc660`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:04:49,022::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:04:49,022::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:04:49,023::task::990::TaskManager.Task::(_decref) Task=`6ff0c07c-be66-4756-8723-e7be044bc660`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:04:49,848::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:49,922::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306892 s, 33.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:04:51,443::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:04:51,935::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:52,013::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0371032 s, 27.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:04:53,920::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:04:53,929::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00172383 s, 2.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:04:54,021::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:54,100::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373988 s, 27.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:04:54,461::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:04:56,113::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:56,549::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.394294 s, 2.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:04:57,043::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1cb1b71a] >Thread-13::DEBUG::2014-05-13 12:04:57,043::task::595::TaskManager.Task::(_updateState) Task=`7edc98d4-f3fd-4d91-989b-ed2c9dc9a246`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:04:57,044::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:04:57,044::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:04:57,044::task::1185::TaskManager.Task::(prepare) Task=`7edc98d4-f3fd-4d91-989b-ed2c9dc9a246`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:04:57,044::task::595::TaskManager.Task::(_updateState) Task=`7edc98d4-f3fd-4d91-989b-ed2c9dc9a246`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:04:57,044::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:04:57,045::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:04:57,045::task::990::TaskManager.Task::(_decref) Task=`7edc98d4-f3fd-4d91-989b-ed2c9dc9a246`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:04:57,049::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1cb1b71a] >Thread-338::DEBUG::2014-05-13 12:04:57,049::task::595::TaskManager.Task::(_updateState) Task=`48418250-35fa-4bec-b220-669d6eace7b8`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:04:57,050::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:04:57,050::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5f63ae82-c446-4ce9-9d70-f199fa35f008`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:04:57,050::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:04:57,051::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:04:57,051::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5f63ae82-c446-4ce9-9d70-f199fa35f008`::Granted request >Thread-338::DEBUG::2014-05-13 12:04:57,052::task::827::TaskManager.Task::(resourceAcquired) Task=`48418250-35fa-4bec-b220-669d6eace7b8`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:04:57,052::task::990::TaskManager.Task::(_decref) Task=`48418250-35fa-4bec-b220-669d6eace7b8`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:04:57,053::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:04:57,053::task::1185::TaskManager.Task::(prepare) Task=`48418250-35fa-4bec-b220-669d6eace7b8`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:04:57,053::task::595::TaskManager.Task::(_updateState) Task=`48418250-35fa-4bec-b220-669d6eace7b8`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:04:57,053::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:04:57,054::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:04:57,054::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:04:57,054::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:04:57,054::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:04:57,054::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:04:57,055::task::990::TaskManager.Task::(_decref) Task=`48418250-35fa-4bec-b220-669d6eace7b8`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:04:57,480::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:04:58,562::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:04:58,645::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0408573 s, 25.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:05:00,499::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:05:00,658::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:00,739::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0378529 s, 27.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:05:02,752::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:02,835::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0386509 s, 26.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:05:03,517::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:05:03,933::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 41830dad-f012-4b4d-b636-863f66d5ca4b >Thread-322::DEBUG::2014-05-13 12:05:03,934::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-322::DEBUG::2014-05-13 12:05:03,937::blockSD::336::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:05:03,948::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.00372689 s, 550 kB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:05:03,948::misc::287::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.00372689 s, 550 kB/s'], size: 2048 >Thread-322::DEBUG::2014-05-13 12:05:03,948::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-322::WARNING::2014-05-13 12:05:03,948::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-322::DEBUG::2014-05-13 12:05:03,949::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-322::DEBUG::2014-05-13 12:05:03,949::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:05:03,949::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:05:03,949::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:05:03,950::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:05:03,950::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:05:03,951::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-13::DEBUG::2014-05-13 12:05:04,152::task::595::TaskManager.Task::(_updateState) Task=`b3a73544-9965-4ce8-b378-fa1022354797`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:05:04,152::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:05:04,153::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00172383', 'lastCheck': '10.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:05:04,153::task::1185::TaskManager.Task::(prepare) Task=`b3a73544-9965-4ce8-b378-fa1022354797`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00172383', 'lastCheck': '10.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:05:04,153::task::595::TaskManager.Task::(_updateState) Task=`b3a73544-9965-4ce8-b378-fa1022354797`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:05:04,153::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:05:04,153::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:05:04,153::task::990::TaskManager.Task::(_decref) Task=`b3a73544-9965-4ce8-b378-fa1022354797`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 12:05:04,178::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:05:04,179::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:05:04,180::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'LOCKPOLICY=', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', 'DESCRIPTION=ISCSI', 'LOGBLKSIZE=512', 'TYPE=ISCSI', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'PHYBLKSIZE=512', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', 'CLASS=Data', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'POOL_DESCRIPTION=DC_2', 'POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'MASTER_VERSION=1', 'ROLE=Master', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', '_SHA_CKSUM=28bb2d7cc60e0d779eb69fe93d050c16c769f069', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0'] >Thread-322::INFO::2014-05-13 12:05:04,181::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_imageNS already registered >Thread-322::INFO::2014-05-13 12:05:04,181::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_volumeNS already registered >Thread-322::INFO::2014-05-13 12:05:04,181::blockSD::452::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_lvmActivationNS already registered >Thread-322::DEBUG::2014-05-13 12:05:04,182::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 12:05:04,350::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:05:04,353::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:05:04,361::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000692588 s, 5.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:05:04,848::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:04,926::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356933 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:05:06,550::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:05:06,939::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:07,018::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363148 s, 28.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:05:07,074::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:05:07,074::task::595::TaskManager.Task::(_updateState) Task=`6d444a85-bea8-4990-aaf2-276cfb50c919`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:05:07,075::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:05:07,075::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:05:07,075::task::1185::TaskManager.Task::(prepare) Task=`6d444a85-bea8-4990-aaf2-276cfb50c919`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:05:07,075::task::595::TaskManager.Task::(_updateState) Task=`6d444a85-bea8-4990-aaf2-276cfb50c919`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:05:07,075::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:05:07,075::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:05:07,076::task::990::TaskManager.Task::(_decref) Task=`6d444a85-bea8-4990-aaf2-276cfb50c919`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:05:07,081::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:05:07,081::task::595::TaskManager.Task::(_updateState) Task=`2ad164b6-59ad-4de5-9dc1-55f715d73a87`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:05:07,081::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:05:07,082::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6f03546c-7e13-42d1-a0d3-f2872f1dc942`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:05:07,082::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:05:07,082::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:05:07,082::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6f03546c-7e13-42d1-a0d3-f2872f1dc942`::Granted request >Thread-338::DEBUG::2014-05-13 12:05:07,083::task::827::TaskManager.Task::(resourceAcquired) Task=`2ad164b6-59ad-4de5-9dc1-55f715d73a87`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:05:07,083::task::990::TaskManager.Task::(_decref) Task=`2ad164b6-59ad-4de5-9dc1-55f715d73a87`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:05:07,083::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:05:07,084::task::1185::TaskManager.Task::(prepare) Task=`2ad164b6-59ad-4de5-9dc1-55f715d73a87`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:05:07,084::task::595::TaskManager.Task::(_updateState) Task=`2ad164b6-59ad-4de5-9dc1-55f715d73a87`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:05:07,084::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:05:07,084::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:05:07,084::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:05:07,084::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:05:07,085::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:05:07,085::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:05:07,085::task::990::TaskManager.Task::(_decref) Task=`2ad164b6-59ad-4de5-9dc1-55f715d73a87`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:05:09,026::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:09,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.35792 s, 2.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:05:09,568::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:05:11,441::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:11,517::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316645 s, 32.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:05:12,586::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:05:13,530::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:13,603::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306131 s, 33.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:05:14,369::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:05:14,377::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00071376 s, 5.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:05:15,605::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:05:15,616::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:15,697::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0370333 s, 27.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:05:17,106::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [72471f8f] >Thread-13::DEBUG::2014-05-13 12:05:17,107::task::595::TaskManager.Task::(_updateState) Task=`7fa49e7d-e501-47e4-aefe-febac56bb422`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:05:17,107::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:05:17,108::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:05:17,108::task::1185::TaskManager.Task::(prepare) Task=`7fa49e7d-e501-47e4-aefe-febac56bb422`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:05:17,108::task::595::TaskManager.Task::(_updateState) Task=`7fa49e7d-e501-47e4-aefe-febac56bb422`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:05:17,108::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:05:17,108::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:05:17,109::task::990::TaskManager.Task::(_decref) Task=`7fa49e7d-e501-47e4-aefe-febac56bb422`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:05:17,113::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [72471f8f] >Thread-338::DEBUG::2014-05-13 12:05:17,114::task::595::TaskManager.Task::(_updateState) Task=`55e68b5b-f63f-4572-89f9-edf64d57cc71`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:05:17,114::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:05:17,114::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`81e452a8-622e-4e56-8484-df87948edbf8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:05:17,115::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:05:17,115::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:05:17,115::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`81e452a8-622e-4e56-8484-df87948edbf8`::Granted request >Thread-338::DEBUG::2014-05-13 12:05:17,115::task::827::TaskManager.Task::(resourceAcquired) Task=`55e68b5b-f63f-4572-89f9-edf64d57cc71`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:05:17,116::task::990::TaskManager.Task::(_decref) Task=`55e68b5b-f63f-4572-89f9-edf64d57cc71`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:05:17,116::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:05:17,116::task::1185::TaskManager.Task::(prepare) Task=`55e68b5b-f63f-4572-89f9-edf64d57cc71`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:05:17,117::task::595::TaskManager.Task::(_updateState) Task=`55e68b5b-f63f-4572-89f9-edf64d57cc71`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:05:17,117::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:05:17,117::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:05:17,117::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:05:17,117::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:05:17,118::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:05:17,118::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:05:17,118::task::990::TaskManager.Task::(_decref) Task=`55e68b5b-f63f-4572-89f9-edf64d57cc71`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:05:17,710::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:17,789::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0371327 s, 27.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:05:18,624::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:05:19,281::task::595::TaskManager.Task::(_updateState) Task=`32672c9a-128b-45b6-bfe0-c6883afefeb0`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:05:19,282::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:05:19,282::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00071376', 'lastCheck': '4.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:05:19,282::task::1185::TaskManager.Task::(prepare) Task=`32672c9a-128b-45b6-bfe0-c6883afefeb0`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00071376', 'lastCheck': '4.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:05:19,282::task::595::TaskManager.Task::(_updateState) Task=`32672c9a-128b-45b6-bfe0-c6883afefeb0`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:05:19,282::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:05:19,282::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:05:19,282::task::990::TaskManager.Task::(_decref) Task=`32672c9a-128b-45b6-bfe0-c6883afefeb0`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:05:19,802::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:19,878::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314298 s, 32.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:05:21,644::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:05:21,891::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:21,983::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0475505 s, 21.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:05:23,996::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:24,070::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0308491 s, 33.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:05:24,403::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:05:24,413::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00175928 s, 2.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:05:24,662::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:05:26,083::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:26,160::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319695 s, 32.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:05:27,136::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:05:27,137::task::595::TaskManager.Task::(_updateState) Task=`810472dd-6752-4e43-9164-dea9c32bf5a9`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:05:27,137::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:05:27,137::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:05:27,138::task::1185::TaskManager.Task::(prepare) Task=`810472dd-6752-4e43-9164-dea9c32bf5a9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:05:27,138::task::595::TaskManager.Task::(_updateState) Task=`810472dd-6752-4e43-9164-dea9c32bf5a9`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:05:27,138::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:05:27,138::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:05:27,138::task::990::TaskManager.Task::(_decref) Task=`810472dd-6752-4e43-9164-dea9c32bf5a9`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:05:27,143::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:05:27,143::task::595::TaskManager.Task::(_updateState) Task=`0c3bca5a-2f57-406f-957d-ca46143d4f54`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:05:27,144::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:05:27,144::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`7f3550aa-c891-470f-8108-2d4a05571cbe`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:05:27,144::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:05:27,144::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:05:27,145::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`7f3550aa-c891-470f-8108-2d4a05571cbe`::Granted request >Thread-338::DEBUG::2014-05-13 12:05:27,145::task::827::TaskManager.Task::(resourceAcquired) Task=`0c3bca5a-2f57-406f-957d-ca46143d4f54`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:05:27,145::task::990::TaskManager.Task::(_decref) Task=`0c3bca5a-2f57-406f-957d-ca46143d4f54`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:05:27,146::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:05:27,146::task::1185::TaskManager.Task::(prepare) Task=`0c3bca5a-2f57-406f-957d-ca46143d4f54`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:05:27,146::task::595::TaskManager.Task::(_updateState) Task=`0c3bca5a-2f57-406f-957d-ca46143d4f54`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:05:27,146::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:05:27,147::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:05:27,147::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:05:27,147::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:05:27,147::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:05:27,147::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:05:27,148::task::990::TaskManager.Task::(_decref) Task=`0c3bca5a-2f57-406f-957d-ca46143d4f54`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:05:27,680::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:05:28,173::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:28,248::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315258 s, 32.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:05:30,261::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:30,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383175 s, 26.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:05:30,698::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:05:32,357::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:32,437::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0368917 s, 27.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:05:33,717::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:05:34,420::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-13::DEBUG::2014-05-13 12:05:34,420::task::595::TaskManager.Task::(_updateState) Task=`666f85b1-6e36-4d07-81a7-5b1db4bad804`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:05:34,427::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:05:34,428::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00175928', 'lastCheck': '10.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:05:34,429::task::1185::TaskManager.Task::(prepare) Task=`666f85b1-6e36-4d07-81a7-5b1db4bad804`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00175928', 'lastCheck': '10.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:05:34,430::task::595::TaskManager.Task::(_updateState) Task=`666f85b1-6e36-4d07-81a7-5b1db4bad804`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:05:34,431::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:05:34,432::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:05:34,434::task::990::TaskManager.Task::(_decref) Task=`666f85b1-6e36-4d07-81a7-5b1db4bad804`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 12:05:34,434::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000511614 s, 8.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:05:34,450::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:34,537::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0414031 s, 24.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:05:36,550::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:36,647::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.05384 s, 19.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:05:36,737::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:05:37,166::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6623fb10] >Thread-13::DEBUG::2014-05-13 12:05:37,166::task::595::TaskManager.Task::(_updateState) Task=`2414240a-526a-4916-960e-bc725547c32b`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:05:37,166::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:05:37,167::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:05:37,167::task::1185::TaskManager.Task::(prepare) Task=`2414240a-526a-4916-960e-bc725547c32b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:05:37,167::task::595::TaskManager.Task::(_updateState) Task=`2414240a-526a-4916-960e-bc725547c32b`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:05:37,167::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:05:37,167::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:05:37,168::task::990::TaskManager.Task::(_decref) Task=`2414240a-526a-4916-960e-bc725547c32b`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:05:37,172::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6623fb10] >Thread-338::DEBUG::2014-05-13 12:05:37,172::task::595::TaskManager.Task::(_updateState) Task=`4840aa53-5fac-4693-b8f3-358fdccdfa18`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:05:37,172::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:05:37,173::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`000c4302-0cb7-45fb-90f2-2056d62f377d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:05:37,173::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:05:37,173::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:05:37,173::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`000c4302-0cb7-45fb-90f2-2056d62f377d`::Granted request >Thread-338::DEBUG::2014-05-13 12:05:37,174::task::827::TaskManager.Task::(resourceAcquired) Task=`4840aa53-5fac-4693-b8f3-358fdccdfa18`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:05:37,174::task::990::TaskManager.Task::(_decref) Task=`4840aa53-5fac-4693-b8f3-358fdccdfa18`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:05:37,174::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:05:37,175::task::1185::TaskManager.Task::(prepare) Task=`4840aa53-5fac-4693-b8f3-358fdccdfa18`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:05:37,175::task::595::TaskManager.Task::(_updateState) Task=`4840aa53-5fac-4693-b8f3-358fdccdfa18`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:05:37,175::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:05:37,175::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:05:37,175::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:05:37,175::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:05:37,176::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:05:37,176::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:05:37,176::task::990::TaskManager.Task::(_decref) Task=`4840aa53-5fac-4693-b8f3-358fdccdfa18`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:05:38,661::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:38,745::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0413585 s, 24.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:05:39,755::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:05:40,756::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:40,833::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0325883 s, 31.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:05:42,775::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:05:42,846::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:42,928::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384266 s, 26.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:05:44,463::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:05:44,471::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000706733 s, 5.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:05:44,941::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:45,022::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387199 s, 26.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:05:45,793::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:05:47,035::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:47,117::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0393925 s, 26.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:05:47,195::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:05:47,195::task::595::TaskManager.Task::(_updateState) Task=`78faad85-2038-4605-a4a2-0be492600c34`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:05:47,196::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:05:47,196::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:05:47,196::task::1185::TaskManager.Task::(prepare) Task=`78faad85-2038-4605-a4a2-0be492600c34`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:05:47,196::task::595::TaskManager.Task::(_updateState) Task=`78faad85-2038-4605-a4a2-0be492600c34`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:05:47,196::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:05:47,197::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:05:47,197::task::990::TaskManager.Task::(_decref) Task=`78faad85-2038-4605-a4a2-0be492600c34`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:05:47,201::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:05:47,202::task::595::TaskManager.Task::(_updateState) Task=`8ad01242-7d58-4693-961c-d66a28ae3f70`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:05:47,202::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:05:47,202::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d3179c70-00c7-4884-bb80-8a23b1962b0e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:05:47,203::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:05:47,203::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:05:47,203::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d3179c70-00c7-4884-bb80-8a23b1962b0e`::Granted request >Thread-338::DEBUG::2014-05-13 12:05:47,203::task::827::TaskManager.Task::(resourceAcquired) Task=`8ad01242-7d58-4693-961c-d66a28ae3f70`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:05:47,204::task::990::TaskManager.Task::(_decref) Task=`8ad01242-7d58-4693-961c-d66a28ae3f70`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:05:47,204::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:05:47,204::task::1185::TaskManager.Task::(prepare) Task=`8ad01242-7d58-4693-961c-d66a28ae3f70`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:05:47,205::task::595::TaskManager.Task::(_updateState) Task=`8ad01242-7d58-4693-961c-d66a28ae3f70`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:05:47,205::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:05:47,205::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:05:47,205::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:05:47,205::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:05:47,206::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:05:47,206::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:05:47,206::task::990::TaskManager.Task::(_decref) Task=`8ad01242-7d58-4693-961c-d66a28ae3f70`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:05:48,811::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:05:49,130::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:49,205::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0336677 s, 30.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:05:49,601::task::595::TaskManager.Task::(_updateState) Task=`d8c3ff63-086a-48e1-b35b-56fb778e2706`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:05:49,601::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:05:49,602::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000706733', 'lastCheck': '5.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:05:49,602::task::1185::TaskManager.Task::(prepare) Task=`d8c3ff63-086a-48e1-b35b-56fb778e2706`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000706733', 'lastCheck': '5.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:05:49,602::task::595::TaskManager.Task::(_updateState) Task=`d8c3ff63-086a-48e1-b35b-56fb778e2706`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:05:49,602::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:05:49,602::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:05:49,602::task::990::TaskManager.Task::(_decref) Task=`d8c3ff63-086a-48e1-b35b-56fb778e2706`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:05:51,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:51,311::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0506486 s, 20.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:05:51,831::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:05:53,324::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:53,399::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0331982 s, 30.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:05:54,479::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:05:54,487::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00070608 s, 5.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:05:54,853::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:05:55,412::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:55,488::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0338516 s, 30.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:05:57,225::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [206acf2a] >Thread-13::DEBUG::2014-05-13 12:05:57,226::task::595::TaskManager.Task::(_updateState) Task=`60027b68-577a-4a73-a9b8-8866dd113dda`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:05:57,226::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:05:57,226::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:05:57,226::task::1185::TaskManager.Task::(prepare) Task=`60027b68-577a-4a73-a9b8-8866dd113dda`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:05:57,227::task::595::TaskManager.Task::(_updateState) Task=`60027b68-577a-4a73-a9b8-8866dd113dda`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:05:57,227::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:05:57,227::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:05:57,227::task::990::TaskManager.Task::(_decref) Task=`60027b68-577a-4a73-a9b8-8866dd113dda`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:05:57,232::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [206acf2a] >Thread-338::DEBUG::2014-05-13 12:05:57,232::task::595::TaskManager.Task::(_updateState) Task=`c0675ad3-7b26-4583-b1bd-753853f8344b`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:05:57,232::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:05:57,233::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1c06a542-9199-42c7-b709-3f2e2215c86d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:05:57,233::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:05:57,233::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:05:57,233::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1c06a542-9199-42c7-b709-3f2e2215c86d`::Granted request >Thread-338::DEBUG::2014-05-13 12:05:57,234::task::827::TaskManager.Task::(resourceAcquired) Task=`c0675ad3-7b26-4583-b1bd-753853f8344b`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:05:57,234::task::990::TaskManager.Task::(_decref) Task=`c0675ad3-7b26-4583-b1bd-753853f8344b`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:05:57,235::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:05:57,235::task::1185::TaskManager.Task::(prepare) Task=`c0675ad3-7b26-4583-b1bd-753853f8344b`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:05:57,235::task::595::TaskManager.Task::(_updateState) Task=`c0675ad3-7b26-4583-b1bd-753853f8344b`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:05:57,235::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:05:57,235::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:05:57,236::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:05:57,236::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:05:57,236::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:05:57,236::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:05:57,236::task::990::TaskManager.Task::(_decref) Task=`c0675ad3-7b26-4583-b1bd-753853f8344b`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:05:57,501::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:57,583::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384269 s, 26.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:05:57,872::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:05:59,596::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:05:59,671::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0331902 s, 30.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:06:00,890::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:06:01,684::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:06:01,760::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0347227 s, 29.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:06:03,773::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:06:03,855::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.038098 s, 26.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:06:03,911::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:06:04,494::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:06:04,501::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00072551 s, 5.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:06:04,732::task::595::TaskManager.Task::(_updateState) Task=`cde017f2-1b13-4aaa-a018-6ace3ac77c2e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:06:04,733::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:06:04,733::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00072551', 'lastCheck': '0.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:06:04,733::task::1185::TaskManager.Task::(prepare) Task=`cde017f2-1b13-4aaa-a018-6ace3ac77c2e`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00072551', 'lastCheck': '0.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:06:04,733::task::595::TaskManager.Task::(_updateState) Task=`cde017f2-1b13-4aaa-a018-6ace3ac77c2e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:06:04,733::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:06:04,734::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:06:04,734::task::990::TaskManager.Task::(_decref) Task=`cde017f2-1b13-4aaa-a018-6ace3ac77c2e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:06:05,868::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:06:05,961::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0498827 s, 20.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:06:06,935::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:06:07,254::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17df19a0] >Thread-13::DEBUG::2014-05-13 12:06:07,255::task::595::TaskManager.Task::(_updateState) Task=`8209084d-2ee6-4f38-91e4-444a29388aa8`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:06:07,255::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:06:07,255::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:06:07,255::task::1185::TaskManager.Task::(prepare) Task=`8209084d-2ee6-4f38-91e4-444a29388aa8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:06:07,255::task::595::TaskManager.Task::(_updateState) Task=`8209084d-2ee6-4f38-91e4-444a29388aa8`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:06:07,256::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:06:07,256::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:06:07,256::task::990::TaskManager.Task::(_decref) Task=`8209084d-2ee6-4f38-91e4-444a29388aa8`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:06:07,261::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17df19a0] >Thread-338::DEBUG::2014-05-13 12:06:07,261::task::595::TaskManager.Task::(_updateState) Task=`6c89499c-768f-49f5-9c67-e939056506f1`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:06:07,261::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:06:07,261::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1454b39d-14b8-4a25-9665-69ff6e4072ed`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:06:07,262::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:06:07,262::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:06:07,262::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1454b39d-14b8-4a25-9665-69ff6e4072ed`::Granted request >Thread-338::DEBUG::2014-05-13 12:06:07,262::task::827::TaskManager.Task::(resourceAcquired) Task=`6c89499c-768f-49f5-9c67-e939056506f1`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:06:07,262::task::990::TaskManager.Task::(_decref) Task=`6c89499c-768f-49f5-9c67-e939056506f1`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:06:07,263::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:06:07,263::task::1185::TaskManager.Task::(prepare) Task=`6c89499c-768f-49f5-9c67-e939056506f1`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:06:07,263::task::595::TaskManager.Task::(_updateState) Task=`6c89499c-768f-49f5-9c67-e939056506f1`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:06:07,263::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:06:07,264::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:06:07,264::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:06:07,264::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:06:07,264::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:06:07,264::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:06:07,265::task::990::TaskManager.Task::(_decref) Task=`6c89499c-768f-49f5-9c67-e939056506f1`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:06:07,973::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:06:08,051::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352739 s, 29.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:06:09,954::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:06:10,064::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:06:10,140::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0339255 s, 30.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:06:12,153::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:06:12,234::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0390275 s, 26.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:06:12,973::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:06:14,247::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:06:14,323::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0335397 s, 30.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:06:14,509::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:06:14,517::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000821204 s, 5.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:06:15,989::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:06:16,336::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:06:16,410::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0325549 s, 31.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:06:17,282::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [12ac301a] >Thread-13::DEBUG::2014-05-13 12:06:17,282::task::595::TaskManager.Task::(_updateState) Task=`be857515-0c27-4555-ac09-6539710cd16f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:06:17,283::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:06:17,283::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:06:17,283::task::1185::TaskManager.Task::(prepare) Task=`be857515-0c27-4555-ac09-6539710cd16f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:06:17,283::task::595::TaskManager.Task::(_updateState) Task=`be857515-0c27-4555-ac09-6539710cd16f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:06:17,284::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:06:17,284::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:06:17,284::task::990::TaskManager.Task::(_decref) Task=`be857515-0c27-4555-ac09-6539710cd16f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:06:17,288::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [12ac301a] >Thread-338::DEBUG::2014-05-13 12:06:17,289::task::595::TaskManager.Task::(_updateState) Task=`c977032d-000e-4e74-bcfd-d0e89b58e0fa`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:06:17,289::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:06:17,289::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`53308ae6-c25b-49fc-9050-dcc81d583677`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:06:17,290::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:06:17,290::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:06:17,290::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`53308ae6-c25b-49fc-9050-dcc81d583677`::Granted request >Thread-338::DEBUG::2014-05-13 12:06:17,290::task::827::TaskManager.Task::(resourceAcquired) Task=`c977032d-000e-4e74-bcfd-d0e89b58e0fa`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:06:17,290::task::990::TaskManager.Task::(_decref) Task=`c977032d-000e-4e74-bcfd-d0e89b58e0fa`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:06:17,291::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:06:17,291::task::1185::TaskManager.Task::(prepare) Task=`c977032d-000e-4e74-bcfd-d0e89b58e0fa`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:06:17,292::task::595::TaskManager.Task::(_updateState) Task=`c977032d-000e-4e74-bcfd-d0e89b58e0fa`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:06:17,292::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:06:17,292::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:06:17,292::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:06:17,292::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:06:17,293::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:06:17,293::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:06:17,293::task::990::TaskManager.Task::(_decref) Task=`c977032d-000e-4e74-bcfd-d0e89b58e0fa`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:06:18,423::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:06:18,517::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0504836 s, 20.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:06:19,008::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:06:19,876::task::595::TaskManager.Task::(_updateState) Task=`f73485eb-80f0-4b20-9e0c-63eb09266823`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:06:19,877::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:06:19,877::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000821204', 'lastCheck': '5.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:06:19,877::task::1185::TaskManager.Task::(prepare) Task=`f73485eb-80f0-4b20-9e0c-63eb09266823`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000821204', 'lastCheck': '5.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:06:19,877::task::595::TaskManager.Task::(_updateState) Task=`f73485eb-80f0-4b20-9e0c-63eb09266823`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:06:19,877::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:06:19,877::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:06:19,877::task::990::TaskManager.Task::(_decref) Task=`f73485eb-80f0-4b20-9e0c-63eb09266823`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:06:20,530::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:06:20,605::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0333196 s, 30.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:06:22,031::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:06:22,618::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:06:22,694::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0342488 s, 29.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:06:24,524::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:06:24,532::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00065011 s, 6.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:06:24,706::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:06:24,785::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.038266 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:06:25,051::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:06:26,798::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:06:26,881::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.040069 s, 25.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:06:27,318::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:06:27,349::task::595::TaskManager.Task::(_updateState) Task=`7b463cef-5e3e-4f6d-8437-63d8c5e852dd`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:06:27,350::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:06:27,350::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:06:27,350::task::1185::TaskManager.Task::(prepare) Task=`7b463cef-5e3e-4f6d-8437-63d8c5e852dd`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:06:27,350::task::595::TaskManager.Task::(_updateState) Task=`7b463cef-5e3e-4f6d-8437-63d8c5e852dd`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:06:27,351::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:06:27,351::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:06:27,351::task::990::TaskManager.Task::(_decref) Task=`7b463cef-5e3e-4f6d-8437-63d8c5e852dd`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:06:27,356::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:06:27,356::task::595::TaskManager.Task::(_updateState) Task=`9dc36650-40b6-42d2-b622-679171952b18`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:06:27,356::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:06:27,357::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fc49f3d3-305d-46a9-a716-a14af89dbe88`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:06:27,357::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:06:27,357::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:06:27,357::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fc49f3d3-305d-46a9-a716-a14af89dbe88`::Granted request >Thread-338::DEBUG::2014-05-13 12:06:27,358::task::827::TaskManager.Task::(resourceAcquired) Task=`9dc36650-40b6-42d2-b622-679171952b18`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:06:27,358::task::990::TaskManager.Task::(_decref) Task=`9dc36650-40b6-42d2-b622-679171952b18`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:06:27,359::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:06:27,359::task::1185::TaskManager.Task::(prepare) Task=`9dc36650-40b6-42d2-b622-679171952b18`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:06:27,359::task::595::TaskManager.Task::(_updateState) Task=`9dc36650-40b6-42d2-b622-679171952b18`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:06:27,359::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:06:27,359::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:06:27,360::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:06:27,360::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:06:27,360::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:06:27,360::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:06:27,360::task::990::TaskManager.Task::(_decref) Task=`9dc36650-40b6-42d2-b622-679171952b18`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:06:28,069::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:06:28,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:06:28,969::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0332955 s, 30.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:06:30,982::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:06:31,063::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384542 s, 26.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:06:31,088::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:06:33,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:06:33,172::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0520861 s, 19.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:06:34,108::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:06:34,539::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:06:34,547::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000736991 s, 5.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:06:35,017::task::595::TaskManager.Task::(_updateState) Task=`ab3084e1-d50e-4db4-8400-c960207e230c`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:06:35,018::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:06:35,018::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000736991', 'lastCheck': '0.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:06:35,018::task::1185::TaskManager.Task::(prepare) Task=`ab3084e1-d50e-4db4-8400-c960207e230c`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000736991', 'lastCheck': '0.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:06:35,018::task::595::TaskManager.Task::(_updateState) Task=`ab3084e1-d50e-4db4-8400-c960207e230c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:06:35,018::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:06:35,019::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:06:35,019::task::990::TaskManager.Task::(_decref) Task=`ab3084e1-d50e-4db4-8400-c960207e230c`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:06:35,185::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:06:35,265::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0388811 s, 26.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:06:37,127::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:06:37,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:06:37,355::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0335712 s, 30.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:06:37,380::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:06:37,380::task::595::TaskManager.Task::(_updateState) Task=`1f39f83a-6e1c-498d-9536-d1dc17c4f032`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:06:37,381::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:06:37,381::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:06:37,381::task::1185::TaskManager.Task::(prepare) Task=`1f39f83a-6e1c-498d-9536-d1dc17c4f032`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:06:37,381::task::595::TaskManager.Task::(_updateState) Task=`1f39f83a-6e1c-498d-9536-d1dc17c4f032`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:06:37,381::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:06:37,382::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:06:37,382::task::990::TaskManager.Task::(_decref) Task=`1f39f83a-6e1c-498d-9536-d1dc17c4f032`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:06:37,386::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:06:37,387::task::595::TaskManager.Task::(_updateState) Task=`36570b78-54d7-4618-876b-009e3c5c068b`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:06:37,387::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:06:37,387::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`45dfe5a6-2229-4fef-93cf-ea7f8cb80284`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:06:37,387::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:06:37,388::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:06:37,388::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`45dfe5a6-2229-4fef-93cf-ea7f8cb80284`::Granted request >Thread-338::DEBUG::2014-05-13 12:06:37,388::task::827::TaskManager.Task::(resourceAcquired) Task=`36570b78-54d7-4618-876b-009e3c5c068b`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:06:37,388::task::990::TaskManager.Task::(_decref) Task=`36570b78-54d7-4618-876b-009e3c5c068b`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:06:37,389::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:06:37,389::task::1185::TaskManager.Task::(prepare) Task=`36570b78-54d7-4618-876b-009e3c5c068b`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:06:37,390::task::595::TaskManager.Task::(_updateState) Task=`36570b78-54d7-4618-876b-009e3c5c068b`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:06:37,390::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:06:37,390::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:06:37,390::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:06:37,390::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:06:37,391::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:06:37,391::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:06:37,391::task::990::TaskManager.Task::(_decref) Task=`36570b78-54d7-4618-876b-009e3c5c068b`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:06:39,368::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:06:39,448::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.03826 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:06:40,146::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:06:41,461::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:06:41,537::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0340304 s, 30.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:06:43,164::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:06:43,550::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:06:43,625::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0327884 s, 31.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:06:44,554::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:06:44,562::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000671656 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:06:45,638::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:06:45,718::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381718 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:06:46,184::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:06:47,410::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:06:47,410::task::595::TaskManager.Task::(_updateState) Task=`5b172160-9b0b-41ff-93cb-b03b1087a192`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:06:47,411::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:06:47,411::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:06:47,411::task::1185::TaskManager.Task::(prepare) Task=`5b172160-9b0b-41ff-93cb-b03b1087a192`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:06:47,411::task::595::TaskManager.Task::(_updateState) Task=`5b172160-9b0b-41ff-93cb-b03b1087a192`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:06:47,411::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:06:47,412::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:06:47,412::task::990::TaskManager.Task::(_decref) Task=`5b172160-9b0b-41ff-93cb-b03b1087a192`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:06:47,416::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:06:47,416::task::595::TaskManager.Task::(_updateState) Task=`42c81b65-167f-4277-a87b-824201ddfab6`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:06:47,417::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:06:47,417::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`19872522-af10-4e9d-af2e-9a1f7f2d499c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:06:47,417::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:06:47,417::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:06:47,418::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`19872522-af10-4e9d-af2e-9a1f7f2d499c`::Granted request >Thread-338::DEBUG::2014-05-13 12:06:47,418::task::827::TaskManager.Task::(resourceAcquired) Task=`42c81b65-167f-4277-a87b-824201ddfab6`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:06:47,418::task::990::TaskManager.Task::(_decref) Task=`42c81b65-167f-4277-a87b-824201ddfab6`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:06:47,419::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:06:47,419::task::1185::TaskManager.Task::(prepare) Task=`42c81b65-167f-4277-a87b-824201ddfab6`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:06:47,419::task::595::TaskManager.Task::(_updateState) Task=`42c81b65-167f-4277-a87b-824201ddfab6`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:06:47,419::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:06:47,420::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:06:47,420::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:06:47,420::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:06:47,420::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:06:47,420::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:06:47,421::task::990::TaskManager.Task::(_decref) Task=`42c81b65-167f-4277-a87b-824201ddfab6`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:06:47,731::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:06:47,815::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.039789 s, 25.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:06:49,214::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:06:49,828::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 12:06:50,150::task::595::TaskManager.Task::(_updateState) Task=`2778a0e1-af5d-4242-937e-29d12d07d76e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:06:50,151::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:06:50,151::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000671656', 'lastCheck': '5.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:06:50,151::task::1185::TaskManager.Task::(prepare) Task=`2778a0e1-af5d-4242-937e-29d12d07d76e`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000671656', 'lastCheck': '5.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:06:50,151::task::595::TaskManager.Task::(_updateState) Task=`2778a0e1-af5d-4242-937e-29d12d07d76e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:06:50,152::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:06:50,152::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:06:50,152::task::990::TaskManager.Task::(_decref) Task=`2778a0e1-af5d-4242-937e-29d12d07d76e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:06:50,200::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.326783 s, 3.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:06:52,213::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:06:52,241::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:06:52,296::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0415353 s, 24.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:06:54,309::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:06:54,389::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0380546 s, 26.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:06:54,570::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:06:54,578::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000719901 s, 5.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:06:55,260::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:06:56,402::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:06:56,479::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0341986 s, 29.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:06:57,440::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [19876d15] >Thread-13::DEBUG::2014-05-13 12:06:57,440::task::595::TaskManager.Task::(_updateState) Task=`ce5a35c5-bb12-423c-ac40-230484033195`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:06:57,441::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:06:57,441::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:06:57,441::task::1185::TaskManager.Task::(prepare) Task=`ce5a35c5-bb12-423c-ac40-230484033195`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:06:57,441::task::595::TaskManager.Task::(_updateState) Task=`ce5a35c5-bb12-423c-ac40-230484033195`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:06:57,442::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:06:57,442::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:06:57,442::task::990::TaskManager.Task::(_decref) Task=`ce5a35c5-bb12-423c-ac40-230484033195`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:06:57,446::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [19876d15] >Thread-338::DEBUG::2014-05-13 12:06:57,447::task::595::TaskManager.Task::(_updateState) Task=`f203d0ce-c6fb-46ab-af5c-4e123a902f81`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:06:57,447::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:06:57,447::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a77c0a22-2fef-4661-a0bd-cf24ab8f5e76`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:06:57,448::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:06:57,448::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:06:57,448::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a77c0a22-2fef-4661-a0bd-cf24ab8f5e76`::Granted request >Thread-338::DEBUG::2014-05-13 12:06:57,448::task::827::TaskManager.Task::(resourceAcquired) Task=`f203d0ce-c6fb-46ab-af5c-4e123a902f81`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:06:57,449::task::990::TaskManager.Task::(_decref) Task=`f203d0ce-c6fb-46ab-af5c-4e123a902f81`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:06:57,449::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:06:57,449::task::1185::TaskManager.Task::(prepare) Task=`f203d0ce-c6fb-46ab-af5c-4e123a902f81`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:06:57,450::task::595::TaskManager.Task::(_updateState) Task=`f203d0ce-c6fb-46ab-af5c-4e123a902f81`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:06:57,450::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:06:57,450::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:06:57,450::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:06:57,450::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:06:57,451::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:06:57,451::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:06:57,451::task::990::TaskManager.Task::(_decref) Task=`f203d0ce-c6fb-46ab-af5c-4e123a902f81`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:06:58,281::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:06:58,492::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:06:58,567::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0333615 s, 30.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:07:00,580::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:00,662::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0385544 s, 26.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:07:01,299::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:07:02,675::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:02,767::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0496958 s, 20.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:07:04,318::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:07:04,584::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:07:04,592::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000699125 s, 5.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:07:04,780::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:04,860::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0371266 s, 27.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:07:05,287::task::595::TaskManager.Task::(_updateState) Task=`8d98a1cd-a3b2-4ce9-88be-437c45bab412`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:07:05,287::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:07:05,287::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000699125', 'lastCheck': '0.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:07:05,287::task::1185::TaskManager.Task::(prepare) Task=`8d98a1cd-a3b2-4ce9-88be-437c45bab412`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000699125', 'lastCheck': '0.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:07:05,288::task::595::TaskManager.Task::(_updateState) Task=`8d98a1cd-a3b2-4ce9-88be-437c45bab412`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:07:05,288::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:07:05,288::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:07:05,288::task::990::TaskManager.Task::(_decref) Task=`8d98a1cd-a3b2-4ce9-88be-437c45bab412`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:07:06,873::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:06,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0390038 s, 26.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:07:07,347::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:07:07,468::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:07:07,468::task::595::TaskManager.Task::(_updateState) Task=`acba23a0-73d1-4eb0-a726-0a28bf4142c4`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:07:07,468::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:07:07,469::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:07:07,469::task::1185::TaskManager.Task::(prepare) Task=`acba23a0-73d1-4eb0-a726-0a28bf4142c4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:07:07,469::task::595::TaskManager.Task::(_updateState) Task=`acba23a0-73d1-4eb0-a726-0a28bf4142c4`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:07:07,469::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:07:07,469::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:07:07,470::task::990::TaskManager.Task::(_decref) Task=`acba23a0-73d1-4eb0-a726-0a28bf4142c4`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:07:07,476::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:07:07,476::task::595::TaskManager.Task::(_updateState) Task=`cfc9b6eb-6e00-4f56-aaa0-0ecca1960ed7`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:07:07,476::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:07:07,477::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`566f91c9-e528-4727-bfd9-aae90c72e865`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:07:07,477::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:07:07,477::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:07:07,477::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`566f91c9-e528-4727-bfd9-aae90c72e865`::Granted request >Thread-338::DEBUG::2014-05-13 12:07:07,478::task::827::TaskManager.Task::(resourceAcquired) Task=`cfc9b6eb-6e00-4f56-aaa0-0ecca1960ed7`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:07:07,478::task::990::TaskManager.Task::(_decref) Task=`cfc9b6eb-6e00-4f56-aaa0-0ecca1960ed7`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:07:07,479::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:07:07,479::task::1185::TaskManager.Task::(prepare) Task=`cfc9b6eb-6e00-4f56-aaa0-0ecca1960ed7`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:07:07,479::task::595::TaskManager.Task::(_updateState) Task=`cfc9b6eb-6e00-4f56-aaa0-0ecca1960ed7`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:07:07,479::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:07:07,479::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:07:07,480::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:07:07,480::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:07:07,480::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:07:07,480::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:07:07,480::task::990::TaskManager.Task::(_decref) Task=`cfc9b6eb-6e00-4f56-aaa0-0ecca1960ed7`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:07:08,980::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:09,060::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373032 s, 27.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:07:10,365::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:07:11,074::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:11,150::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316134 s, 32.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:07:13,163::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:13,236::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309619 s, 33.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:07:13,382::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:07:14,599::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:07:14,608::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000736199 s, 5.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:07:15,250::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:15,331::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384108 s, 26.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:07:16,401::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:07:17,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:17,441::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.049467 s, 20.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:07:17,499::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [30b3ccf] >Thread-13::DEBUG::2014-05-13 12:07:17,499::task::595::TaskManager.Task::(_updateState) Task=`b18cbeed-b795-4fd2-ac6d-5e4e5af1d5dc`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:07:17,500::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:07:17,500::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:07:17,500::task::1185::TaskManager.Task::(prepare) Task=`b18cbeed-b795-4fd2-ac6d-5e4e5af1d5dc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:07:17,500::task::595::TaskManager.Task::(_updateState) Task=`b18cbeed-b795-4fd2-ac6d-5e4e5af1d5dc`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:07:17,501::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:07:17,501::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:07:17,501::task::990::TaskManager.Task::(_decref) Task=`b18cbeed-b795-4fd2-ac6d-5e4e5af1d5dc`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:07:17,506::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [30b3ccf] >Thread-338::DEBUG::2014-05-13 12:07:17,507::task::595::TaskManager.Task::(_updateState) Task=`a3ee6fd8-b6d7-4921-b2dd-f8c346e6c672`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:07:17,507::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:07:17,508::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d08098e6-034b-47fe-95ec-5fe77bbeecf9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:07:17,509::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:07:17,509::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:07:17,509::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d08098e6-034b-47fe-95ec-5fe77bbeecf9`::Granted request >Thread-338::DEBUG::2014-05-13 12:07:17,509::task::827::TaskManager.Task::(resourceAcquired) Task=`a3ee6fd8-b6d7-4921-b2dd-f8c346e6c672`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:07:17,510::task::990::TaskManager.Task::(_decref) Task=`a3ee6fd8-b6d7-4921-b2dd-f8c346e6c672`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:07:17,510::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:07:17,510::task::1185::TaskManager.Task::(prepare) Task=`a3ee6fd8-b6d7-4921-b2dd-f8c346e6c672`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:07:17,511::task::595::TaskManager.Task::(_updateState) Task=`a3ee6fd8-b6d7-4921-b2dd-f8c346e6c672`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:07:17,511::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:07:17,511::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:07:17,511::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:07:17,511::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:07:17,512::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:07:17,512::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:07:17,512::task::990::TaskManager.Task::(_decref) Task=`a3ee6fd8-b6d7-4921-b2dd-f8c346e6c672`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:07:19,421::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:07:19,454::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:19,528::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305597 s, 33.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:07:20,421::task::595::TaskManager.Task::(_updateState) Task=`14ee72c3-8bff-45f1-b2e8-15dda4b7d1d2`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:07:20,421::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:07:20,421::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000736199', 'lastCheck': '5.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:07:20,421::task::1185::TaskManager.Task::(prepare) Task=`14ee72c3-8bff-45f1-b2e8-15dda4b7d1d2`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000736199', 'lastCheck': '5.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:07:20,421::task::595::TaskManager.Task::(_updateState) Task=`14ee72c3-8bff-45f1-b2e8-15dda4b7d1d2`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:07:20,422::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:07:20,422::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:07:20,422::task::990::TaskManager.Task::(_decref) Task=`14ee72c3-8bff-45f1-b2e8-15dda4b7d1d2`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:07:21,541::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:21,628::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.041958 s, 24.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:07:22,440::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:07:23,641::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:23,720::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353067 s, 29.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:07:24,615::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:07:24,624::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000683746 s, 6.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:07:25,460::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:07:25,733::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:25,808::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319931 s, 32.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:07:27,531::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:07:27,531::task::595::TaskManager.Task::(_updateState) Task=`d8d53738-116b-4cf9-9b75-fd40e403cf48`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:07:27,531::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:07:27,532::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:07:27,532::task::1185::TaskManager.Task::(prepare) Task=`d8d53738-116b-4cf9-9b75-fd40e403cf48`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:07:27,532::task::595::TaskManager.Task::(_updateState) Task=`d8d53738-116b-4cf9-9b75-fd40e403cf48`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:07:27,532::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:07:27,533::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:07:27,533::task::990::TaskManager.Task::(_decref) Task=`d8d53738-116b-4cf9-9b75-fd40e403cf48`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:07:27,537::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:07:27,537::task::595::TaskManager.Task::(_updateState) Task=`b66e9380-34e8-4f31-abb9-7578e5663445`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:07:27,538::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:07:27,538::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8a976503-9f72-4f5e-bc63-581fccc60159`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:07:27,538::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:07:27,538::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:07:27,539::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8a976503-9f72-4f5e-bc63-581fccc60159`::Granted request >Thread-338::DEBUG::2014-05-13 12:07:27,539::task::827::TaskManager.Task::(resourceAcquired) Task=`b66e9380-34e8-4f31-abb9-7578e5663445`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:07:27,539::task::990::TaskManager.Task::(_decref) Task=`b66e9380-34e8-4f31-abb9-7578e5663445`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:07:27,540::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:07:27,540::task::1185::TaskManager.Task::(prepare) Task=`b66e9380-34e8-4f31-abb9-7578e5663445`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:07:27,540::task::595::TaskManager.Task::(_updateState) Task=`b66e9380-34e8-4f31-abb9-7578e5663445`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:07:27,540::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:07:27,541::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:07:27,541::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:07:27,541::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:07:27,541::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:07:27,541::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:07:27,542::task::990::TaskManager.Task::(_decref) Task=`b66e9380-34e8-4f31-abb9-7578e5663445`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:07:27,821::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:27,901::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0372468 s, 27.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:07:28,481::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:07:29,914::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:29,990::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316567 s, 32.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:07:31,499::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:07:32,003::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:32,374::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.328534 s, 3.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:07:34,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:34,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0340336 s, 30.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:07:34,518::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:07:34,631::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:07:34,641::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00286134 s, 1.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:07:35,555::task::595::TaskManager.Task::(_updateState) Task=`a61bba5d-3db6-45ad-8243-95fd10fbef8c`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:07:35,555::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:07:35,556::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00286134', 'lastCheck': '0.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:07:35,556::task::1185::TaskManager.Task::(prepare) Task=`a61bba5d-3db6-45ad-8243-95fd10fbef8c`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00286134', 'lastCheck': '0.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:07:35,556::task::595::TaskManager.Task::(_updateState) Task=`a61bba5d-3db6-45ad-8243-95fd10fbef8c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:07:35,556::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:07:35,556::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:07:35,556::task::990::TaskManager.Task::(_decref) Task=`a61bba5d-3db6-45ad-8243-95fd10fbef8c`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:07:36,476::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:36,558::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0391392 s, 26.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:07:37,543::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:07:37,562::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [7d725256] >Thread-13::DEBUG::2014-05-13 12:07:37,563::task::595::TaskManager.Task::(_updateState) Task=`ddf611ce-ced8-433a-b47e-3747f8c33886`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:07:37,563::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:07:37,563::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:07:37,563::task::1185::TaskManager.Task::(prepare) Task=`ddf611ce-ced8-433a-b47e-3747f8c33886`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:07:37,564::task::595::TaskManager.Task::(_updateState) Task=`ddf611ce-ced8-433a-b47e-3747f8c33886`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:07:37,564::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:07:37,564::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:07:37,564::task::990::TaskManager.Task::(_decref) Task=`ddf611ce-ced8-433a-b47e-3747f8c33886`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:07:37,568::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [7d725256] >Thread-338::DEBUG::2014-05-13 12:07:37,569::task::595::TaskManager.Task::(_updateState) Task=`735690c6-d5f3-4b72-b807-9f49955b0be8`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:07:37,569::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:07:37,569::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e1bced58-67cf-4945-9cc0-9da8c6470157`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:07:37,569::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:07:37,570::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:07:37,570::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e1bced58-67cf-4945-9cc0-9da8c6470157`::Granted request >Thread-338::DEBUG::2014-05-13 12:07:37,570::task::827::TaskManager.Task::(resourceAcquired) Task=`735690c6-d5f3-4b72-b807-9f49955b0be8`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:07:37,570::task::990::TaskManager.Task::(_decref) Task=`735690c6-d5f3-4b72-b807-9f49955b0be8`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:07:37,571::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:07:37,571::task::1185::TaskManager.Task::(prepare) Task=`735690c6-d5f3-4b72-b807-9f49955b0be8`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:07:37,571::task::595::TaskManager.Task::(_updateState) Task=`735690c6-d5f3-4b72-b807-9f49955b0be8`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:07:37,572::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:07:37,572::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:07:37,572::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:07:37,572::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:07:37,572::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:07:37,573::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:07:37,573::task::990::TaskManager.Task::(_decref) Task=`735690c6-d5f3-4b72-b807-9f49955b0be8`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:07:38,571::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:38,658::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384359 s, 26.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:07:40,563::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:07:40,671::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:41,114::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.401571 s, 2.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:07:43,128::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:43,203::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383239 s, 26.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:07:43,583::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:07:44,647::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:07:44,655::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000702198 s, 5.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:07:45,211::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:45,290::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352881 s, 29.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:07:46,604::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:07:47,302::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:47,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031981 s, 32.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:07:47,594::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [12ac301a] >Thread-13::DEBUG::2014-05-13 12:07:47,594::task::595::TaskManager.Task::(_updateState) Task=`eb31fd92-9871-438c-b108-b4cededc5194`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:07:47,595::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:07:47,595::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:07:47,595::task::1185::TaskManager.Task::(prepare) Task=`eb31fd92-9871-438c-b108-b4cededc5194`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:07:47,595::task::595::TaskManager.Task::(_updateState) Task=`eb31fd92-9871-438c-b108-b4cededc5194`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:07:47,596::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:07:47,596::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:07:47,596::task::990::TaskManager.Task::(_decref) Task=`eb31fd92-9871-438c-b108-b4cededc5194`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:07:47,916::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [12ac301a] >Thread-338::DEBUG::2014-05-13 12:07:47,916::task::595::TaskManager.Task::(_updateState) Task=`0447e513-98d9-4673-aac4-3c4d1e9cd027`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:07:47,916::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:07:47,917::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`758d52b7-552b-448c-b70f-8ff417674871`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:07:47,917::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:07:47,917::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:07:47,917::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`758d52b7-552b-448c-b70f-8ff417674871`::Granted request >Thread-338::DEBUG::2014-05-13 12:07:47,918::task::827::TaskManager.Task::(resourceAcquired) Task=`0447e513-98d9-4673-aac4-3c4d1e9cd027`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:07:47,918::task::990::TaskManager.Task::(_decref) Task=`0447e513-98d9-4673-aac4-3c4d1e9cd027`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:07:47,919::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:07:47,919::task::1185::TaskManager.Task::(prepare) Task=`0447e513-98d9-4673-aac4-3c4d1e9cd027`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:07:47,919::task::595::TaskManager.Task::(_updateState) Task=`0447e513-98d9-4673-aac4-3c4d1e9cd027`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:07:47,919::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:07:47,919::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:07:47,920::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:07:47,920::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:07:47,920::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:07:47,920::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:07:47,920::task::990::TaskManager.Task::(_decref) Task=`0447e513-98d9-4673-aac4-3c4d1e9cd027`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:07:49,399::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:49,474::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313623 s, 32.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:07:49,623::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:07:50,924::task::595::TaskManager.Task::(_updateState) Task=`eaac52b5-3cf8-4a00-9913-fca0b2f2918c`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:07:50,924::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:07:50,924::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000702198', 'lastCheck': '6.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:07:50,925::task::1185::TaskManager.Task::(prepare) Task=`eaac52b5-3cf8-4a00-9913-fca0b2f2918c`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000702198', 'lastCheck': '6.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:07:50,925::task::595::TaskManager.Task::(_updateState) Task=`eaac52b5-3cf8-4a00-9913-fca0b2f2918c`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:07:50,925::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:07:50,925::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:07:50,925::task::990::TaskManager.Task::(_decref) Task=`eaac52b5-3cf8-4a00-9913-fca0b2f2918c`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:07:51,487::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:51,568::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0376047 s, 27.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:07:52,642::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:07:53,581::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:53,662::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037034 s, 27.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:07:54,663::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:07:54,671::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00074955 s, 5.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:07:55,665::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:07:55,676::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:55,762::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.043277 s, 23.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:07:57,775::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:57,855::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359058 s, 28.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:07:57,940::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [28fd42b9] >Thread-66::DEBUG::2014-05-13 12:07:57,941::task::595::TaskManager.Task::(_updateState) Task=`3ce391f5-2b42-4f65-a747-74f20cc36652`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:07:57,941::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:07:57,941::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:07:57,942::task::1185::TaskManager.Task::(prepare) Task=`3ce391f5-2b42-4f65-a747-74f20cc36652`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:07:57,942::task::595::TaskManager.Task::(_updateState) Task=`3ce391f5-2b42-4f65-a747-74f20cc36652`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:07:57,942::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:07:57,942::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:07:57,942::task::990::TaskManager.Task::(_decref) Task=`3ce391f5-2b42-4f65-a747-74f20cc36652`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:07:57,947::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [28fd42b9] >Thread-338::DEBUG::2014-05-13 12:07:57,948::task::595::TaskManager.Task::(_updateState) Task=`7fb58bf0-d347-47c8-9eb7-008c5b16637c`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:07:57,948::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:07:57,948::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`bc311261-818e-4e7b-9b8c-bed8117347e2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:07:57,948::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:07:57,949::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:07:57,949::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`bc311261-818e-4e7b-9b8c-bed8117347e2`::Granted request >Thread-338::DEBUG::2014-05-13 12:07:57,949::task::827::TaskManager.Task::(resourceAcquired) Task=`7fb58bf0-d347-47c8-9eb7-008c5b16637c`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:07:57,949::task::990::TaskManager.Task::(_decref) Task=`7fb58bf0-d347-47c8-9eb7-008c5b16637c`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:07:57,950::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:07:57,950::task::1185::TaskManager.Task::(prepare) Task=`7fb58bf0-d347-47c8-9eb7-008c5b16637c`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:07:57,951::task::595::TaskManager.Task::(_updateState) Task=`7fb58bf0-d347-47c8-9eb7-008c5b16637c`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:07:57,951::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:07:57,951::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:07:57,951::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:07:57,951::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:07:57,952::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:07:57,952::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:07:57,952::task::990::TaskManager.Task::(_decref) Task=`7fb58bf0-d347-47c8-9eb7-008c5b16637c`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:07:58,682::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:07:59,868::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:07:59,944::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0330208 s, 31.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:08:01,701::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:08:01,957::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:02,033::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316642 s, 32.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:08:04,046::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:04,125::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359718 s, 28.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:08:04,678::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:08:04,686::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00083732 s, 4.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:08:04,720::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:08:06,052::task::595::TaskManager.Task::(_updateState) Task=`1a5f6604-9b56-475b-bc2d-d0e96fae37b4`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:08:06,053::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:08:06,053::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00083732', 'lastCheck': '1.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:08:06,053::task::1185::TaskManager.Task::(prepare) Task=`1a5f6604-9b56-475b-bc2d-d0e96fae37b4`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00083732', 'lastCheck': '1.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:08:06,053::task::595::TaskManager.Task::(_updateState) Task=`1a5f6604-9b56-475b-bc2d-d0e96fae37b4`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:08:06,053::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:08:06,054::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:08:06,054::task::990::TaskManager.Task::(_decref) Task=`1a5f6604-9b56-475b-bc2d-d0e96fae37b4`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:08:06,138::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:06,217::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364465 s, 28.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:08:07,740::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:08:07,971::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:08:07,971::task::595::TaskManager.Task::(_updateState) Task=`ca8656c3-2ec0-4541-ad02-84bed7482c5f`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:08:07,972::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:08:07,972::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:08:07,972::task::1185::TaskManager.Task::(prepare) Task=`ca8656c3-2ec0-4541-ad02-84bed7482c5f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:08:07,972::task::595::TaskManager.Task::(_updateState) Task=`ca8656c3-2ec0-4541-ad02-84bed7482c5f`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:08:07,972::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:08:07,973::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:08:07,973::task::990::TaskManager.Task::(_decref) Task=`ca8656c3-2ec0-4541-ad02-84bed7482c5f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:08:07,977::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:08:07,977::task::595::TaskManager.Task::(_updateState) Task=`851f3e15-4b3e-4bd9-b8a4-b569436ee098`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:08:07,977::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:08:07,978::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`79909e99-56dd-499f-8bdd-5a6d88fc8009`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:08:07,978::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:08:07,979::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:08:07,979::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`79909e99-56dd-499f-8bdd-5a6d88fc8009`::Granted request >Thread-338::DEBUG::2014-05-13 12:08:07,979::task::827::TaskManager.Task::(resourceAcquired) Task=`851f3e15-4b3e-4bd9-b8a4-b569436ee098`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:08:07,979::task::990::TaskManager.Task::(_decref) Task=`851f3e15-4b3e-4bd9-b8a4-b569436ee098`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:08:07,980::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:08:07,980::task::1185::TaskManager.Task::(prepare) Task=`851f3e15-4b3e-4bd9-b8a4-b569436ee098`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:08:07,980::task::595::TaskManager.Task::(_updateState) Task=`851f3e15-4b3e-4bd9-b8a4-b569436ee098`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:08:07,981::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:08:07,981::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:08:07,981::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:08:07,981::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:08:07,981::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:08:07,982::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:08:07,982::task::990::TaskManager.Task::(_decref) Task=`851f3e15-4b3e-4bd9-b8a4-b569436ee098`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:08:08,231::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:08,311::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379391 s, 27.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:08:10,324::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:10,676::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.310142 s, 3.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:08:10,759::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:08:12,689::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:12,770::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361286 s, 28.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:08:13,777::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:08:14,693::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:08:14,701::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00062749 s, 6.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:08:14,783::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:14,863::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0369479 s, 27.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:08:16,796::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:08:16,876::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:16,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.032741 s, 31.3 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:08:17,999::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:08:17,999::task::595::TaskManager.Task::(_updateState) Task=`a661207a-4168-4802-9e2a-0155ff66d2e8`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:08:18,000::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:08:18,000::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:08:18,000::task::1185::TaskManager.Task::(prepare) Task=`a661207a-4168-4802-9e2a-0155ff66d2e8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:08:18,000::task::595::TaskManager.Task::(_updateState) Task=`a661207a-4168-4802-9e2a-0155ff66d2e8`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:08:18,000::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:08:18,001::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:08:18,001::task::990::TaskManager.Task::(_decref) Task=`a661207a-4168-4802-9e2a-0155ff66d2e8`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:08:18,006::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:08:18,006::task::595::TaskManager.Task::(_updateState) Task=`6bcc8bb0-eb55-45cd-865c-03220b96924a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:08:18,006::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:08:18,007::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8417ab88-3fa4-4473-b5d8-71dbfad6b827`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:08:18,007::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:08:18,007::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:08:18,007::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8417ab88-3fa4-4473-b5d8-71dbfad6b827`::Granted request >Thread-338::DEBUG::2014-05-13 12:08:18,008::task::827::TaskManager.Task::(resourceAcquired) Task=`6bcc8bb0-eb55-45cd-865c-03220b96924a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:08:18,008::task::990::TaskManager.Task::(_decref) Task=`6bcc8bb0-eb55-45cd-865c-03220b96924a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:08:18,009::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:08:18,009::task::1185::TaskManager.Task::(prepare) Task=`6bcc8bb0-eb55-45cd-865c-03220b96924a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:08:18,009::task::595::TaskManager.Task::(_updateState) Task=`6bcc8bb0-eb55-45cd-865c-03220b96924a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:08:18,009::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:08:18,010::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:08:18,010::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:08:18,010::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:08:18,010::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:08:18,010::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:08:18,011::task::990::TaskManager.Task::(_decref) Task=`6bcc8bb0-eb55-45cd-865c-03220b96924a`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:08:18,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:19,046::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0390536 s, 26.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:08:19,813::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:08:21,059::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:21,158::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0431384 s, 23.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:08:21,180::task::595::TaskManager.Task::(_updateState) Task=`e6c6d634-3613-4462-9992-9bc2190b04bc`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:08:21,180::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:08:21,180::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00062749', 'lastCheck': '6.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:08:21,180::task::1185::TaskManager.Task::(prepare) Task=`e6c6d634-3613-4462-9992-9bc2190b04bc`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00062749', 'lastCheck': '6.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:08:21,180::task::595::TaskManager.Task::(_updateState) Task=`e6c6d634-3613-4462-9992-9bc2190b04bc`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:08:21,181::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:08:21,181::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:08:21,181::task::990::TaskManager.Task::(_decref) Task=`e6c6d634-3613-4462-9992-9bc2190b04bc`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:08:22,834::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:08:23,171::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:23,253::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365007 s, 28.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:08:24,708::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:08:24,716::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000562626 s, 7.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:08:25,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:25,341::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320605 s, 31.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:08:25,853::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:08:27,354::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:27,436::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384274 s, 26.6 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:08:28,029::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:08:28,038::task::595::TaskManager.Task::(_updateState) Task=`0cc268bd-a29a-40c3-b606-ba5c6bd79507`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:08:28,038::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:08:28,039::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:08:28,039::task::1185::TaskManager.Task::(prepare) Task=`0cc268bd-a29a-40c3-b606-ba5c6bd79507`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:08:28,039::task::595::TaskManager.Task::(_updateState) Task=`0cc268bd-a29a-40c3-b606-ba5c6bd79507`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:08:28,039::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:08:28,039::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:08:28,040::task::990::TaskManager.Task::(_decref) Task=`0cc268bd-a29a-40c3-b606-ba5c6bd79507`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:08:28,044::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:08:28,044::task::595::TaskManager.Task::(_updateState) Task=`e76ecb99-ce63-489f-9b09-da5bcba0cc52`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:08:28,045::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:08:28,045::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c83cd1ea-2ec3-4e64-97a9-03dd07028343`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:08:28,045::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:08:28,046::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:08:28,046::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c83cd1ea-2ec3-4e64-97a9-03dd07028343`::Granted request >Thread-338::DEBUG::2014-05-13 12:08:28,047::task::827::TaskManager.Task::(resourceAcquired) Task=`e76ecb99-ce63-489f-9b09-da5bcba0cc52`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:08:28,047::task::990::TaskManager.Task::(_decref) Task=`e76ecb99-ce63-489f-9b09-da5bcba0cc52`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:08:28,048::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:08:28,048::task::1185::TaskManager.Task::(prepare) Task=`e76ecb99-ce63-489f-9b09-da5bcba0cc52`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:08:28,049::task::595::TaskManager.Task::(_updateState) Task=`e76ecb99-ce63-489f-9b09-da5bcba0cc52`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:08:28,049::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:08:28,049::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:08:28,049::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:08:28,049::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:08:28,049::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:08:28,050::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:08:28,050::task::990::TaskManager.Task::(_decref) Task=`e76ecb99-ce63-489f-9b09-da5bcba0cc52`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:08:28,871::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:08:29,449::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:29,524::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319317 s, 32.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:08:31,537::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:31,615::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0346311 s, 29.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:08:31,890::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:08:33,628::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:33,709::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382108 s, 26.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:08:34,723::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:08:34,731::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000732889 s, 5.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:08:34,913::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:08:35,722::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:35,802::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036059 s, 28.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:08:36,313::task::595::TaskManager.Task::(_updateState) Task=`e3bb9017-3e85-4457-825f-3fca3c8c551b`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:08:36,313::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:08:36,314::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000732889', 'lastCheck': '1.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:08:36,314::task::1185::TaskManager.Task::(prepare) Task=`e3bb9017-3e85-4457-825f-3fca3c8c551b`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000732889', 'lastCheck': '1.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:08:36,314::task::595::TaskManager.Task::(_updateState) Task=`e3bb9017-3e85-4457-825f-3fca3c8c551b`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:08:36,314::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:08:36,314::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:08:36,315::task::990::TaskManager.Task::(_decref) Task=`e3bb9017-3e85-4457-825f-3fca3c8c551b`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:08:37,815::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:37,907::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0497506 s, 20.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:08:37,932::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:08:38,068::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:08:38,068::task::595::TaskManager.Task::(_updateState) Task=`060daafc-6838-4b2d-97ef-d84f5c7a7474`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:08:38,068::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:08:38,069::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:08:38,069::task::1185::TaskManager.Task::(prepare) Task=`060daafc-6838-4b2d-97ef-d84f5c7a7474`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:08:38,069::task::595::TaskManager.Task::(_updateState) Task=`060daafc-6838-4b2d-97ef-d84f5c7a7474`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:08:38,069::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:08:38,069::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:08:38,070::task::990::TaskManager.Task::(_decref) Task=`060daafc-6838-4b2d-97ef-d84f5c7a7474`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:08:38,074::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:08:38,074::task::595::TaskManager.Task::(_updateState) Task=`2006e1f5-1f3d-42f6-a2e6-940704fa3aaa`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:08:38,074::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:08:38,075::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`32e551b4-860f-41fd-91f2-3b9740238488`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:08:38,075::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:08:38,075::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:08:38,076::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`32e551b4-860f-41fd-91f2-3b9740238488`::Granted request >Thread-338::DEBUG::2014-05-13 12:08:38,076::task::827::TaskManager.Task::(resourceAcquired) Task=`2006e1f5-1f3d-42f6-a2e6-940704fa3aaa`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:08:38,076::task::990::TaskManager.Task::(_decref) Task=`2006e1f5-1f3d-42f6-a2e6-940704fa3aaa`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:08:38,077::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:08:38,077::task::1185::TaskManager.Task::(prepare) Task=`2006e1f5-1f3d-42f6-a2e6-940704fa3aaa`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:08:38,077::task::595::TaskManager.Task::(_updateState) Task=`2006e1f5-1f3d-42f6-a2e6-940704fa3aaa`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:08:38,077::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:08:38,078::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:08:38,078::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:08:38,078::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:08:38,078::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:08:38,078::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:08:38,079::task::990::TaskManager.Task::(_decref) Task=`2006e1f5-1f3d-42f6-a2e6-940704fa3aaa`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:08:39,920::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:40,001::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0392371 s, 26.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:08:40,951::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:08:42,014::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:42,088::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307446 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:08:43,971::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:08:44,101::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:44,176::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317235 s, 32.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:08:44,738::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:08:44,749::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.0035288 s, 1.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:08:46,189::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:46,263::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036169 s, 28.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:08:46,989::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:08:48,096::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:08:48,097::task::595::TaskManager.Task::(_updateState) Task=`431ecdf4-14ab-4ce9-9223-10dc8a008098`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:08:48,097::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:08:48,097::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:08:48,097::task::1185::TaskManager.Task::(prepare) Task=`431ecdf4-14ab-4ce9-9223-10dc8a008098`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:08:48,098::task::595::TaskManager.Task::(_updateState) Task=`431ecdf4-14ab-4ce9-9223-10dc8a008098`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:08:48,098::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:08:48,098::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:08:48,098::task::990::TaskManager.Task::(_decref) Task=`431ecdf4-14ab-4ce9-9223-10dc8a008098`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:08:48,103::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:08:48,103::task::595::TaskManager.Task::(_updateState) Task=`9951894a-7455-4953-9859-bb858a258d56`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:08:48,103::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:08:48,104::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6fcc1858-5a21-49d0-adf5-b19f94d9d4d3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:08:48,104::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:08:48,104::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:08:48,104::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6fcc1858-5a21-49d0-adf5-b19f94d9d4d3`::Granted request >Thread-338::DEBUG::2014-05-13 12:08:48,105::task::827::TaskManager.Task::(resourceAcquired) Task=`9951894a-7455-4953-9859-bb858a258d56`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:08:48,105::task::990::TaskManager.Task::(_decref) Task=`9951894a-7455-4953-9859-bb858a258d56`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:08:48,105::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:08:48,106::task::1185::TaskManager.Task::(prepare) Task=`9951894a-7455-4953-9859-bb858a258d56`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:08:48,106::task::595::TaskManager.Task::(_updateState) Task=`9951894a-7455-4953-9859-bb858a258d56`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:08:48,106::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:08:48,106::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:08:48,106::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:08:48,107::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:08:48,107::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:08:48,107::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:08:48,107::task::990::TaskManager.Task::(_decref) Task=`9951894a-7455-4953-9859-bb858a258d56`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:08:48,272::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:48,354::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373072 s, 27.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:08:50,008::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:08:50,367::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:50,453::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0441233 s, 23.2 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:08:51,456::task::595::TaskManager.Task::(_updateState) Task=`48d7f0e0-da14-4165-a550-4f290724a5e3`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:08:51,456::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:08:51,456::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0035288', 'lastCheck': '6.7', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:08:51,457::task::1185::TaskManager.Task::(prepare) Task=`48d7f0e0-da14-4165-a550-4f290724a5e3`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0035288', 'lastCheck': '6.7', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:08:51,457::task::595::TaskManager.Task::(_updateState) Task=`48d7f0e0-da14-4165-a550-4f290724a5e3`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:08:51,457::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:08:51,457::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:08:51,457::task::990::TaskManager.Task::(_decref) Task=`48d7f0e0-da14-4165-a550-4f290724a5e3`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:08:52,466::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:52,542::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322588 s, 31.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:08:53,029::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:08:54,555::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:54,635::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356049 s, 28.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:08:54,756::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:08:54,764::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000606276 s, 6.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:08:56,048::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:08:56,649::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:56,725::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0324815 s, 31.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:08:58,125::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:08:58,125::task::595::TaskManager.Task::(_updateState) Task=`6925f890-103d-4c33-afdd-64e7d63fddc7`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:08:58,126::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:08:58,126::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:08:58,126::task::1185::TaskManager.Task::(prepare) Task=`6925f890-103d-4c33-afdd-64e7d63fddc7`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:08:58,126::task::595::TaskManager.Task::(_updateState) Task=`6925f890-103d-4c33-afdd-64e7d63fddc7`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:08:58,127::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:08:58,127::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:08:58,127::task::990::TaskManager.Task::(_decref) Task=`6925f890-103d-4c33-afdd-64e7d63fddc7`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:08:58,132::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:08:58,132::task::595::TaskManager.Task::(_updateState) Task=`eb95580e-b0b2-47a8-92a8-805f19201ae8`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:08:58,133::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:08:58,133::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`45e72df7-f43a-4cbb-b59d-a90d5cc7b90f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:08:58,133::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:08:58,134::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:08:58,134::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`45e72df7-f43a-4cbb-b59d-a90d5cc7b90f`::Granted request >Thread-338::DEBUG::2014-05-13 12:08:58,134::task::827::TaskManager.Task::(resourceAcquired) Task=`eb95580e-b0b2-47a8-92a8-805f19201ae8`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:08:58,134::task::990::TaskManager.Task::(_decref) Task=`eb95580e-b0b2-47a8-92a8-805f19201ae8`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:08:58,135::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:08:58,135::task::1185::TaskManager.Task::(prepare) Task=`eb95580e-b0b2-47a8-92a8-805f19201ae8`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:08:58,135::task::595::TaskManager.Task::(_updateState) Task=`eb95580e-b0b2-47a8-92a8-805f19201ae8`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:08:58,136::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:08:58,136::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:08:58,136::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:08:58,136::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:08:58,136::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:08:58,137::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:08:58,137::task::990::TaskManager.Task::(_decref) Task=`eb95580e-b0b2-47a8-92a8-805f19201ae8`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:08:58,738::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:08:58,816::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359611 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:08:59,068::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:09:00,829::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:09:00,910::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0367402 s, 27.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:09:02,086::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:09:02,923::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:09:03,003::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362987 s, 28.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:09:04,771::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:09:04,779::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000613389 s, 6.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:09:05,016::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:09:05,105::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:09:05,385::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.326627 s, 3.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:09:06,589::task::595::TaskManager.Task::(_updateState) Task=`4116e724-2422-42bb-a8fe-1f9884f54d91`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:09:06,589::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:09:06,589::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000613389', 'lastCheck': '1.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:09:06,590::task::1185::TaskManager.Task::(prepare) Task=`4116e724-2422-42bb-a8fe-1f9884f54d91`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000613389', 'lastCheck': '1.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:09:06,590::task::595::TaskManager.Task::(_updateState) Task=`4116e724-2422-42bb-a8fe-1f9884f54d91`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:09:06,590::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:09:06,590::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:09:06,590::task::990::TaskManager.Task::(_decref) Task=`4116e724-2422-42bb-a8fe-1f9884f54d91`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:09:07,398::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:09:07,472::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313485 s, 32.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:09:08,134::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:09:08,155::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4478bd69] >Thread-66::DEBUG::2014-05-13 12:09:08,156::task::595::TaskManager.Task::(_updateState) Task=`e750df9c-283d-40b6-884d-2e9560e65934`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:09:08,156::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:09:08,156::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:09:08,156::task::1185::TaskManager.Task::(prepare) Task=`e750df9c-283d-40b6-884d-2e9560e65934`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:09:08,156::task::595::TaskManager.Task::(_updateState) Task=`e750df9c-283d-40b6-884d-2e9560e65934`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:09:08,157::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:09:08,157::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:09:08,157::task::990::TaskManager.Task::(_decref) Task=`e750df9c-283d-40b6-884d-2e9560e65934`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:09:08,161::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4478bd69] >Thread-338::DEBUG::2014-05-13 12:09:08,162::task::595::TaskManager.Task::(_updateState) Task=`b59fb5a6-a331-494c-8882-bc088d1efc70`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:09:08,162::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:09:08,162::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`bcfb6781-8a81-4bf6-88d2-01bbec534739`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:09:08,163::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:09:08,163::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:09:08,163::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`bcfb6781-8a81-4bf6-88d2-01bbec534739`::Granted request >Thread-338::DEBUG::2014-05-13 12:09:08,163::task::827::TaskManager.Task::(resourceAcquired) Task=`b59fb5a6-a331-494c-8882-bc088d1efc70`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:09:08,164::task::990::TaskManager.Task::(_decref) Task=`b59fb5a6-a331-494c-8882-bc088d1efc70`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:09:08,164::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:09:08,165::task::1185::TaskManager.Task::(prepare) Task=`b59fb5a6-a331-494c-8882-bc088d1efc70`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:09:08,165::task::595::TaskManager.Task::(_updateState) Task=`b59fb5a6-a331-494c-8882-bc088d1efc70`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:09:08,165::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:09:08,165::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:09:08,165::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:09:08,166::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:09:08,166::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:09:08,166::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:09:08,166::task::990::TaskManager.Task::(_decref) Task=`b59fb5a6-a331-494c-8882-bc088d1efc70`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:09:09,485::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:09:09,800::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.223024 s, 4.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:09:11,155::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:09:11,814::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:09:11,889::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315169 s, 32.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:09:13,902::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:09:13,978::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319839 s, 32.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:09:14,175::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:09:14,786::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:09:14,793::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000734093 s, 5.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:09:15,991::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:09:16,073::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0375773 s, 27.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:09:17,192::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:09:18,086::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 12:09:18,184::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17998964] >Thread-66::DEBUG::2014-05-13 12:09:18,185::task::595::TaskManager.Task::(_updateState) Task=`f66e8f2f-826b-49db-83e2-ea86cc8e1839`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:09:18,185::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:09:18,185::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:09:18,185::task::1185::TaskManager.Task::(prepare) Task=`f66e8f2f-826b-49db-83e2-ea86cc8e1839`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:09:18,185::task::595::TaskManager.Task::(_updateState) Task=`f66e8f2f-826b-49db-83e2-ea86cc8e1839`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:09:18,186::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:09:18,186::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:09:18,186::task::990::TaskManager.Task::(_decref) Task=`f66e8f2f-826b-49db-83e2-ea86cc8e1839`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:09:18,190::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17998964] >Thread-338::DEBUG::2014-05-13 12:09:18,191::task::595::TaskManager.Task::(_updateState) Task=`32611520-9125-4bf0-abe8-2bc4f117bd74`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:09:18,191::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:09:18,191::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3004d254-e8c7-457e-a4ac-2b28a207527d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:09:18,192::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:09:18,192::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:09:18,192::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3004d254-e8c7-457e-a4ac-2b28a207527d`::Granted request >Thread-338::DEBUG::2014-05-13 12:09:18,193::task::827::TaskManager.Task::(resourceAcquired) Task=`32611520-9125-4bf0-abe8-2bc4f117bd74`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:09:18,193::task::990::TaskManager.Task::(_decref) Task=`32611520-9125-4bf0-abe8-2bc4f117bd74`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:09:18,193::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:09:18,194::task::1185::TaskManager.Task::(prepare) Task=`32611520-9125-4bf0-abe8-2bc4f117bd74`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:09:18,194::task::595::TaskManager.Task::(_updateState) Task=`32611520-9125-4bf0-abe8-2bc4f117bd74`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:09:18,194::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:09:18,194::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:09:18,195::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:09:18,195::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:09:18,195::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:09:18,195::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:09:18,195::task::990::TaskManager.Task::(_decref) Task=`32611520-9125-4bf0-abe8-2bc4f117bd74`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:09:18,539::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.411499 s, 2.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:09:20,211::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:09:20,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:09:20,627::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314006 s, 32.6 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:09:21,729::task::595::TaskManager.Task::(_updateState) Task=`3e118242-8e9f-4ef3-b7b6-f7b38c0a5934`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:09:21,729::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:09:21,730::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000734093', 'lastCheck': '6.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:09:21,730::task::1185::TaskManager.Task::(prepare) Task=`3e118242-8e9f-4ef3-b7b6-f7b38c0a5934`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000734093', 'lastCheck': '6.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:09:21,730::task::595::TaskManager.Task::(_updateState) Task=`3e118242-8e9f-4ef3-b7b6-f7b38c0a5934`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:09:21,730::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:09:21,730::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:09:21,730::task::990::TaskManager.Task::(_decref) Task=`3e118242-8e9f-4ef3-b7b6-f7b38c0a5934`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:09:22,640::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:09:22,716::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317806 s, 32.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:09:23,228::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:09:24,729::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 12:09:24,825::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:09:24,843::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000702516 s, 5.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:09:24,867::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0366471 s, 27.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:09:26,245::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:09:26,880::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:09:26,956::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318707 s, 32.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:09:28,213::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:09:28,214::task::595::TaskManager.Task::(_updateState) Task=`d6165f42-ef23-4d20-b899-d64b7f684cf7`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:09:28,214::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:09:28,215::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:09:28,215::task::1185::TaskManager.Task::(prepare) Task=`d6165f42-ef23-4d20-b899-d64b7f684cf7`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:09:28,215::task::595::TaskManager.Task::(_updateState) Task=`d6165f42-ef23-4d20-b899-d64b7f684cf7`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:09:28,215::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:09:28,215::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:09:28,215::task::990::TaskManager.Task::(_decref) Task=`d6165f42-ef23-4d20-b899-d64b7f684cf7`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:09:28,220::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:09:28,220::task::595::TaskManager.Task::(_updateState) Task=`d18dbe66-6992-4be1-a3e5-ad346485c2c0`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:09:28,220::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:09:28,221::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c94f68fb-e9a4-4ed5-8346-1d910afebd5f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:09:28,221::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:09:28,221::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:09:28,222::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c94f68fb-e9a4-4ed5-8346-1d910afebd5f`::Granted request >Thread-338::DEBUG::2014-05-13 12:09:28,222::task::827::TaskManager.Task::(resourceAcquired) Task=`d18dbe66-6992-4be1-a3e5-ad346485c2c0`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:09:28,222::task::990::TaskManager.Task::(_decref) Task=`d18dbe66-6992-4be1-a3e5-ad346485c2c0`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:09:28,223::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:09:28,223::task::1185::TaskManager.Task::(prepare) Task=`d18dbe66-6992-4be1-a3e5-ad346485c2c0`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:09:28,223::task::595::TaskManager.Task::(_updateState) Task=`d18dbe66-6992-4be1-a3e5-ad346485c2c0`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:09:28,223::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:09:28,224::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:09:28,224::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:09:28,224::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:09:28,224::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:09:28,224::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:09:28,225::task::990::TaskManager.Task::(_decref) Task=`d18dbe66-6992-4be1-a3e5-ad346485c2c0`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:09:28,969::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:09:29,045::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0325293 s, 31.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:09:29,264::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:09:31,058::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:09:31,508::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.408573 s, 2.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:09:32,287::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:09:33,521::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:09:33,601::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383986 s, 26.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:09:34,862::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:09:34,870::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000608606 s, 6.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:09:35,304::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:09:35,615::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:09:35,697::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.038578 s, 26.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:09:36,865::task::595::TaskManager.Task::(_updateState) Task=`7abdc3e5-0a86-4ce5-9979-851295b34a81`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:09:36,890::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:09:36,890::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000608606', 'lastCheck': '2.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:09:36,890::task::1185::TaskManager.Task::(prepare) Task=`7abdc3e5-0a86-4ce5-9979-851295b34a81`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000608606', 'lastCheck': '2.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:09:36,890::task::595::TaskManager.Task::(_updateState) Task=`7abdc3e5-0a86-4ce5-9979-851295b34a81`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:09:36,890::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:09:36,891::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:09:36,891::task::990::TaskManager.Task::(_decref) Task=`7abdc3e5-0a86-4ce5-9979-851295b34a81`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:09:37,710::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:09:37,784::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322267 s, 31.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:09:38,243::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5eff0b16] >Thread-66::DEBUG::2014-05-13 12:09:38,243::task::595::TaskManager.Task::(_updateState) Task=`1ac611f1-6dd0-4a11-ae91-5dc5cb5c788a`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:09:38,244::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:09:38,244::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:09:38,244::task::1185::TaskManager.Task::(prepare) Task=`1ac611f1-6dd0-4a11-ae91-5dc5cb5c788a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:09:38,244::task::595::TaskManager.Task::(_updateState) Task=`1ac611f1-6dd0-4a11-ae91-5dc5cb5c788a`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:09:38,245::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:09:38,245::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:09:38,245::task::990::TaskManager.Task::(_decref) Task=`1ac611f1-6dd0-4a11-ae91-5dc5cb5c788a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:09:38,250::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5eff0b16] >Thread-338::DEBUG::2014-05-13 12:09:38,250::task::595::TaskManager.Task::(_updateState) Task=`4a77d75e-f77c-4b61-988f-6750805c4ef8`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:09:38,250::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:09:38,251::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ac7358ee-d02a-4562-8b28-0ee254c03bb1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:09:38,251::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:09:38,252::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:09:38,252::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ac7358ee-d02a-4562-8b28-0ee254c03bb1`::Granted request >Thread-338::DEBUG::2014-05-13 12:09:38,253::task::827::TaskManager.Task::(resourceAcquired) Task=`4a77d75e-f77c-4b61-988f-6750805c4ef8`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:09:38,253::task::990::TaskManager.Task::(_decref) Task=`4a77d75e-f77c-4b61-988f-6750805c4ef8`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:09:38,254::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:09:38,254::task::1185::TaskManager.Task::(prepare) Task=`4a77d75e-f77c-4b61-988f-6750805c4ef8`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:09:38,254::task::595::TaskManager.Task::(_updateState) Task=`4a77d75e-f77c-4b61-988f-6750805c4ef8`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:09:38,254::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:09:38,254::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:09:38,255::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:09:38,255::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:09:38,255::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:09:38,255::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:09:38,255::task::990::TaskManager.Task::(_decref) Task=`4a77d75e-f77c-4b61-988f-6750805c4ef8`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:09:38,323::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:09:39,797::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:09:40,068::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0742974 s, 13.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:09:41,342::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:09:42,080::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:09:42,155::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314027 s, 32.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:09:44,168::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:09:44,365::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:09:44,530::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.318831 s, 3.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:09:44,877::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:09:44,885::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00081725 s, 5.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:09:46,543::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:09:46,624::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357584 s, 28.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:09:47,383::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:09:48,278::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:09:48,278::task::595::TaskManager.Task::(_updateState) Task=`c3c9c472-e603-4e47-86ca-058ec20ed4e7`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:09:48,278::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:09:48,279::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:09:48,279::task::1185::TaskManager.Task::(prepare) Task=`c3c9c472-e603-4e47-86ca-058ec20ed4e7`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:09:48,279::task::595::TaskManager.Task::(_updateState) Task=`c3c9c472-e603-4e47-86ca-058ec20ed4e7`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:09:48,279::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:09:48,279::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:09:48,280::task::990::TaskManager.Task::(_decref) Task=`c3c9c472-e603-4e47-86ca-058ec20ed4e7`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:09:48,285::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:09:48,286::task::595::TaskManager.Task::(_updateState) Task=`e6a9da59-584b-49bc-a021-8282eb780745`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:09:48,286::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:09:48,286::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`400e09d1-0bb8-4a34-b99f-3c356e533f29`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:09:48,287::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:09:48,287::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:09:48,287::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`400e09d1-0bb8-4a34-b99f-3c356e533f29`::Granted request >Thread-338::DEBUG::2014-05-13 12:09:48,287::task::827::TaskManager.Task::(resourceAcquired) Task=`e6a9da59-584b-49bc-a021-8282eb780745`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:09:48,288::task::990::TaskManager.Task::(_decref) Task=`e6a9da59-584b-49bc-a021-8282eb780745`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:09:48,288::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:09:48,288::task::1185::TaskManager.Task::(prepare) Task=`e6a9da59-584b-49bc-a021-8282eb780745`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:09:48,289::task::595::TaskManager.Task::(_updateState) Task=`e6a9da59-584b-49bc-a021-8282eb780745`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:09:48,289::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:09:48,289::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:09:48,289::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:09:48,289::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:09:48,290::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:09:48,290::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:09:48,290::task::990::TaskManager.Task::(_decref) Task=`e6a9da59-584b-49bc-a021-8282eb780745`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:09:48,637::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:09:48,720::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0395872 s, 25.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:09:50,412::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:09:50,732::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:09:50,808::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322621 s, 31.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:09:52,032::task::595::TaskManager.Task::(_updateState) Task=`f1ead6c2-74ac-4a33-9ee1-0ff87992f15a`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:09:52,033::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:09:52,033::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00081725', 'lastCheck': '7.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:09:52,033::task::1185::TaskManager.Task::(prepare) Task=`f1ead6c2-74ac-4a33-9ee1-0ff87992f15a`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00081725', 'lastCheck': '7.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:09:52,033::task::595::TaskManager.Task::(_updateState) Task=`f1ead6c2-74ac-4a33-9ee1-0ff87992f15a`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:09:52,033::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:09:52,034::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:09:52,034::task::990::TaskManager.Task::(_decref) Task=`f1ead6c2-74ac-4a33-9ee1-0ff87992f15a`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:09:52,821::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:09:52,906::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0324114 s, 31.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:09:53,439::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:09:54,892::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:09:54,901::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000764587 s, 5.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:09:54,920::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:09:55,001::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0392515 s, 26.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:09:56,462::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:09:57,014::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:09:57,428::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.369758 s, 2.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:09:58,312::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2555e502] >Thread-66::DEBUG::2014-05-13 12:09:58,313::task::595::TaskManager.Task::(_updateState) Task=`429fedf5-73c2-4266-9e25-b54006a85c4f`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:09:58,313::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:09:58,313::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:09:58,313::task::1185::TaskManager.Task::(prepare) Task=`429fedf5-73c2-4266-9e25-b54006a85c4f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:09:58,314::task::595::TaskManager.Task::(_updateState) Task=`429fedf5-73c2-4266-9e25-b54006a85c4f`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:09:58,314::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:09:58,314::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:09:58,314::task::990::TaskManager.Task::(_decref) Task=`429fedf5-73c2-4266-9e25-b54006a85c4f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:09:58,318::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2555e502] >Thread-338::DEBUG::2014-05-13 12:09:58,318::task::595::TaskManager.Task::(_updateState) Task=`e84c4be2-0745-4648-bafc-8163f12f3c48`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:09:58,319::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:09:58,319::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4e02797c-e9d9-4254-95e8-d04230b7e74e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:09:58,319::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:09:58,320::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:09:58,320::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4e02797c-e9d9-4254-95e8-d04230b7e74e`::Granted request >Thread-338::DEBUG::2014-05-13 12:09:58,320::task::827::TaskManager.Task::(resourceAcquired) Task=`e84c4be2-0745-4648-bafc-8163f12f3c48`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:09:58,320::task::990::TaskManager.Task::(_decref) Task=`e84c4be2-0745-4648-bafc-8163f12f3c48`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:09:58,321::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:09:58,321::task::1185::TaskManager.Task::(prepare) Task=`e84c4be2-0745-4648-bafc-8163f12f3c48`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:09:58,321::task::595::TaskManager.Task::(_updateState) Task=`e84c4be2-0745-4648-bafc-8163f12f3c48`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:09:58,322::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:09:58,322::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:09:58,322::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:09:58,322::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:09:58,322::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:09:58,322::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:09:58,323::task::990::TaskManager.Task::(_decref) Task=`e84c4be2-0745-4648-bafc-8163f12f3c48`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:09:59,441::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:09:59,488::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:09:59,519::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315661 s, 32.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:10:01,532::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:01,608::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0333703 s, 30.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:10:02,514::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:10:03,621::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:03,704::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0391803 s, 26.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:10:04,905::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 41830dad-f012-4b4d-b636-863f66d5ca4b >Thread-322::DEBUG::2014-05-13 12:10:04,906::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-322::DEBUG::2014-05-13 12:10:04,908::blockSD::336::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:10:04,916::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000718849 s, 2.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:10:04,917::misc::287::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000718849 s, 2.8 MB/s'], size: 2048 >Thread-322::DEBUG::2014-05-13 12:10:04,917::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-322::WARNING::2014-05-13 12:10:04,917::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-322::DEBUG::2014-05-13 12:10:04,917::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-322::DEBUG::2014-05-13 12:10:04,918::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:10:04,918::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:10:04,918::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:10:04,918::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:10:04,919::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:10:04,920::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 12:10:05,190::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:10:05,191::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:10:05,192::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'LOCKPOLICY=', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', 'DESCRIPTION=ISCSI', 'LOGBLKSIZE=512', 'TYPE=ISCSI', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'PHYBLKSIZE=512', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', 'CLASS=Data', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'POOL_DESCRIPTION=DC_2', 'POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'MASTER_VERSION=1', 'ROLE=Master', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', '_SHA_CKSUM=28bb2d7cc60e0d779eb69fe93d050c16c769f069', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0'] >Thread-322::INFO::2014-05-13 12:10:05,193::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_imageNS already registered >Thread-322::INFO::2014-05-13 12:10:05,193::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_volumeNS already registered >Thread-322::INFO::2014-05-13 12:10:05,193::blockSD::452::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_lvmActivationNS already registered >Thread-322::DEBUG::2014-05-13 12:10:05,194::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 12:10:05,403::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:10:05,406::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:10:05,414::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000639742 s, 6.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:10:05,533::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:10:05,717::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:05,800::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387369 s, 26.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:10:07,172::task::595::TaskManager.Task::(_updateState) Task=`f620f679-fce0-4514-997c-91f8caa86243`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:10:07,173::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:10:07,173::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000639742', 'lastCheck': '1.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:10:07,173::task::1185::TaskManager.Task::(prepare) Task=`f620f679-fce0-4514-997c-91f8caa86243`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000639742', 'lastCheck': '1.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:10:07,173::task::595::TaskManager.Task::(_updateState) Task=`f620f679-fce0-4514-997c-91f8caa86243`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:10:07,173::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:10:07,174::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:10:07,174::task::990::TaskManager.Task::(_decref) Task=`f620f679-fce0-4514-997c-91f8caa86243`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:10:07,812::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:08,181::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.325035 s, 3.2 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:10:08,342::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1cb1b71a] >Thread-66::DEBUG::2014-05-13 12:10:08,342::task::595::TaskManager.Task::(_updateState) Task=`3e7dcf8f-7842-4685-b328-5037798659d5`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:10:08,343::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:10:08,343::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:10:08,343::task::1185::TaskManager.Task::(prepare) Task=`3e7dcf8f-7842-4685-b328-5037798659d5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:10:08,343::task::595::TaskManager.Task::(_updateState) Task=`3e7dcf8f-7842-4685-b328-5037798659d5`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:10:08,344::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:10:08,344::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:10:08,344::task::990::TaskManager.Task::(_decref) Task=`3e7dcf8f-7842-4685-b328-5037798659d5`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:10:08,348::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1cb1b71a] >Thread-338::DEBUG::2014-05-13 12:10:08,349::task::595::TaskManager.Task::(_updateState) Task=`38b5b23a-5a6d-4447-b853-b1f53416949d`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:10:08,349::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:10:08,349::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`05728dc4-9c83-4052-8224-461d71c1c257`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:10:08,350::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:10:08,350::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:10:08,350::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`05728dc4-9c83-4052-8224-461d71c1c257`::Granted request >Thread-338::DEBUG::2014-05-13 12:10:08,350::task::827::TaskManager.Task::(resourceAcquired) Task=`38b5b23a-5a6d-4447-b853-b1f53416949d`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:10:08,351::task::990::TaskManager.Task::(_decref) Task=`38b5b23a-5a6d-4447-b853-b1f53416949d`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:10:08,351::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:10:08,352::task::1185::TaskManager.Task::(prepare) Task=`38b5b23a-5a6d-4447-b853-b1f53416949d`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:10:08,352::task::595::TaskManager.Task::(_updateState) Task=`38b5b23a-5a6d-4447-b853-b1f53416949d`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:10:08,352::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:10:08,352::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:10:08,352::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:10:08,353::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:10:08,353::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:10:08,353::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:10:08,353::task::990::TaskManager.Task::(_decref) Task=`38b5b23a-5a6d-4447-b853-b1f53416949d`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:10:08,556::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:10:10,194::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:10,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0375887 s, 27.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:10:11,575::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:10:12,292::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:12,368::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320175 s, 32.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:10:14,381::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:14,456::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314952 s, 32.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:10:14,602::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:10:15,422::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:10:15,430::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000758167 s, 5.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:10:16,469::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:16,544::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313925 s, 32.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:10:17,625::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:10:18,371::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [7a000b17] >Thread-66::DEBUG::2014-05-13 12:10:18,371::task::595::TaskManager.Task::(_updateState) Task=`cdd54b05-cdd6-4996-90af-f26afbb347b5`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:10:18,372::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:10:18,372::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:10:18,372::task::1185::TaskManager.Task::(prepare) Task=`cdd54b05-cdd6-4996-90af-f26afbb347b5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:10:18,372::task::595::TaskManager.Task::(_updateState) Task=`cdd54b05-cdd6-4996-90af-f26afbb347b5`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:10:18,372::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:10:18,373::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:10:18,373::task::990::TaskManager.Task::(_decref) Task=`cdd54b05-cdd6-4996-90af-f26afbb347b5`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:10:18,378::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [7a000b17] >Thread-338::DEBUG::2014-05-13 12:10:18,378::task::595::TaskManager.Task::(_updateState) Task=`79325d43-0198-4ef4-9628-d4a2bfd52933`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:10:18,379::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:10:18,379::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`89390161-a2ad-4f49-b8f6-88860b7df3e2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:10:18,379::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:10:18,380::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:10:18,380::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`89390161-a2ad-4f49-b8f6-88860b7df3e2`::Granted request >Thread-338::DEBUG::2014-05-13 12:10:18,380::task::827::TaskManager.Task::(resourceAcquired) Task=`79325d43-0198-4ef4-9628-d4a2bfd52933`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:10:18,380::task::990::TaskManager.Task::(_decref) Task=`79325d43-0198-4ef4-9628-d4a2bfd52933`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:10:18,381::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:10:18,381::task::1185::TaskManager.Task::(prepare) Task=`79325d43-0198-4ef4-9628-d4a2bfd52933`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:10:18,381::task::595::TaskManager.Task::(_updateState) Task=`79325d43-0198-4ef4-9628-d4a2bfd52933`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:10:18,382::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:10:18,382::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:10:18,382::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:10:18,382::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:10:18,382::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:10:18,383::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:10:18,383::task::990::TaskManager.Task::(_decref) Task=`79325d43-0198-4ef4-9628-d4a2bfd52933`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:10:18,557::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:18,636::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353068 s, 29.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:10:20,642::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:10:20,649::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:20,724::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312547 s, 32.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:10:22,316::task::595::TaskManager.Task::(_updateState) Task=`81e87a8a-d419-4aac-bce1-88f364d0a177`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:10:22,316::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:10:22,316::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000758167', 'lastCheck': '6.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:10:22,316::task::1185::TaskManager.Task::(prepare) Task=`81e87a8a-d419-4aac-bce1-88f364d0a177`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000758167', 'lastCheck': '6.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:10:22,317::task::595::TaskManager.Task::(_updateState) Task=`81e87a8a-d419-4aac-bce1-88f364d0a177`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:10:22,317::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:10:22,317::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:10:22,317::task::990::TaskManager.Task::(_decref) Task=`81e87a8a-d419-4aac-bce1-88f364d0a177`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:10:22,737::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:22,831::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0516299 s, 19.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:10:23,662::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:10:24,844::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:24,924::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362571 s, 28.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:10:25,437::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:10:25,444::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000658169 s, 6.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:10:26,681::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:10:26,937::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:27,016::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379612 s, 27.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:10:28,401::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:10:28,406::task::595::TaskManager.Task::(_updateState) Task=`74724251-ad96-47fd-a8ac-75a8bb009fde`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:10:28,406::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:10:28,407::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:10:28,407::task::1185::TaskManager.Task::(prepare) Task=`74724251-ad96-47fd-a8ac-75a8bb009fde`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:10:28,407::task::595::TaskManager.Task::(_updateState) Task=`74724251-ad96-47fd-a8ac-75a8bb009fde`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:10:28,407::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:10:28,407::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:10:28,407::task::990::TaskManager.Task::(_decref) Task=`74724251-ad96-47fd-a8ac-75a8bb009fde`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:10:28,412::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:10:28,412::task::595::TaskManager.Task::(_updateState) Task=`2ee21a31-d9cd-41ea-bb37-08cfd120ca8d`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:10:28,412::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:10:28,413::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5025f038-cc19-49fc-b1c7-7fc207f876b7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:10:28,413::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:10:28,413::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:10:28,413::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5025f038-cc19-49fc-b1c7-7fc207f876b7`::Granted request >Thread-338::DEBUG::2014-05-13 12:10:28,414::task::827::TaskManager.Task::(resourceAcquired) Task=`2ee21a31-d9cd-41ea-bb37-08cfd120ca8d`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:10:28,414::task::990::TaskManager.Task::(_decref) Task=`2ee21a31-d9cd-41ea-bb37-08cfd120ca8d`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:10:28,415::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:10:28,415::task::1185::TaskManager.Task::(prepare) Task=`2ee21a31-d9cd-41ea-bb37-08cfd120ca8d`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:10:28,415::task::595::TaskManager.Task::(_updateState) Task=`2ee21a31-d9cd-41ea-bb37-08cfd120ca8d`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:10:28,415::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:10:28,415::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:10:28,416::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:10:28,416::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:10:28,416::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:10:28,416::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:10:28,416::task::990::TaskManager.Task::(_decref) Task=`2ee21a31-d9cd-41ea-bb37-08cfd120ca8d`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:10:29,029::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:29,102::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310258 s, 33.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:10:29,699::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:10:31,115::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:31,195::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363809 s, 28.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:10:32,718::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:10:33,208::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:33,286::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357283 s, 28.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:10:35,299::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:35,385::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0442238 s, 23.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:10:35,454::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:10:35,469::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000815394 s, 5.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:10:35,738::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:10:37,398::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:37,473::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314694 s, 32.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:10:37,510::task::595::TaskManager.Task::(_updateState) Task=`efad04b0-d3f9-4d86-a5b3-3ba140dc542d`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:10:37,510::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:10:37,510::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000815394', 'lastCheck': '2.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:10:37,511::task::1185::TaskManager.Task::(prepare) Task=`efad04b0-d3f9-4d86-a5b3-3ba140dc542d`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000815394', 'lastCheck': '2.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:10:37,511::task::595::TaskManager.Task::(_updateState) Task=`efad04b0-d3f9-4d86-a5b3-3ba140dc542d`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:10:37,511::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:10:37,511::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:10:37,511::task::990::TaskManager.Task::(_decref) Task=`efad04b0-d3f9-4d86-a5b3-3ba140dc542d`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:10:38,441::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:10:38,442::task::595::TaskManager.Task::(_updateState) Task=`f5db9eeb-c174-4c8b-8b6b-e398c05bf689`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:10:38,442::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:10:38,442::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:10:38,442::task::1185::TaskManager.Task::(prepare) Task=`f5db9eeb-c174-4c8b-8b6b-e398c05bf689`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:10:38,443::task::595::TaskManager.Task::(_updateState) Task=`f5db9eeb-c174-4c8b-8b6b-e398c05bf689`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:10:38,443::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:10:38,443::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:10:38,443::task::990::TaskManager.Task::(_decref) Task=`f5db9eeb-c174-4c8b-8b6b-e398c05bf689`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:10:38,448::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:10:38,448::task::595::TaskManager.Task::(_updateState) Task=`ddebf01d-3b03-4935-8cc6-88780ea198ff`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:10:38,449::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:10:38,449::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b089a536-73ac-42a8-b423-18a4228b8748`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:10:38,449::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:10:38,450::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:10:38,450::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b089a536-73ac-42a8-b423-18a4228b8748`::Granted request >Thread-338::DEBUG::2014-05-13 12:10:38,450::task::827::TaskManager.Task::(resourceAcquired) Task=`ddebf01d-3b03-4935-8cc6-88780ea198ff`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:10:38,450::task::990::TaskManager.Task::(_decref) Task=`ddebf01d-3b03-4935-8cc6-88780ea198ff`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:10:38,451::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:10:38,451::task::1185::TaskManager.Task::(prepare) Task=`ddebf01d-3b03-4935-8cc6-88780ea198ff`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:10:38,452::task::595::TaskManager.Task::(_updateState) Task=`ddebf01d-3b03-4935-8cc6-88780ea198ff`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:10:38,452::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:10:38,452::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:10:38,452::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:10:38,452::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:10:38,453::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:10:38,453::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:10:38,453::task::990::TaskManager.Task::(_decref) Task=`ddebf01d-3b03-4935-8cc6-88780ea198ff`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:10:38,757::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:10:39,499::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:39,580::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.03741 s, 27.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:10:41,593::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:41,668::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312256 s, 32.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:10:41,782::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:10:43,681::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:43,757::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0340419 s, 30.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:10:44,802::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:10:45,480::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:10:45,488::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000640521 s, 6.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:10:45,771::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:45,850::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359014 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:10:47,825::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:10:47,863::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:48,275::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.369269 s, 2.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:10:48,471::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6623fb10] >Thread-66::DEBUG::2014-05-13 12:10:48,472::task::595::TaskManager.Task::(_updateState) Task=`82c8b195-bb67-4ca4-bfaf-c3dd76223ce1`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:10:48,472::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:10:48,473::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:10:48,473::task::1185::TaskManager.Task::(prepare) Task=`82c8b195-bb67-4ca4-bfaf-c3dd76223ce1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:10:48,473::task::595::TaskManager.Task::(_updateState) Task=`82c8b195-bb67-4ca4-bfaf-c3dd76223ce1`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:10:48,473::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:10:48,473::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:10:48,473::task::990::TaskManager.Task::(_decref) Task=`82c8b195-bb67-4ca4-bfaf-c3dd76223ce1`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:10:48,478::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6623fb10] >Thread-338::DEBUG::2014-05-13 12:10:48,478::task::595::TaskManager.Task::(_updateState) Task=`ee1cccd3-d62b-4b29-b33a-71dd0abb7eb9`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:10:48,478::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:10:48,480::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`cfefb325-f53d-42ad-bf63-12bade9561a9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:10:48,480::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:10:48,480::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:10:48,480::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`cfefb325-f53d-42ad-bf63-12bade9561a9`::Granted request >Thread-338::DEBUG::2014-05-13 12:10:48,481::task::827::TaskManager.Task::(resourceAcquired) Task=`ee1cccd3-d62b-4b29-b33a-71dd0abb7eb9`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:10:48,481::task::990::TaskManager.Task::(_decref) Task=`ee1cccd3-d62b-4b29-b33a-71dd0abb7eb9`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:10:48,482::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:10:48,482::task::1185::TaskManager.Task::(prepare) Task=`ee1cccd3-d62b-4b29-b33a-71dd0abb7eb9`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:10:48,482::task::595::TaskManager.Task::(_updateState) Task=`ee1cccd3-d62b-4b29-b33a-71dd0abb7eb9`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:10:48,482::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:10:48,483::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:10:48,483::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:10:48,483::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:10:48,483::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:10:48,483::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:10:48,484::task::990::TaskManager.Task::(_decref) Task=`ee1cccd3-d62b-4b29-b33a-71dd0abb7eb9`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:10:50,288::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:50,361::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306927 s, 33.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:10:50,843::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:10:52,374::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:52,451::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0324708 s, 31.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:10:52,653::task::595::TaskManager.Task::(_updateState) Task=`5df812e9-b3ce-474d-bdcf-210f783d9c8d`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:10:52,653::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:10:52,653::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000640521', 'lastCheck': '7.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:10:52,653::task::1185::TaskManager.Task::(prepare) Task=`5df812e9-b3ce-474d-bdcf-210f783d9c8d`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000640521', 'lastCheck': '7.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:10:52,653::task::595::TaskManager.Task::(_updateState) Task=`5df812e9-b3ce-474d-bdcf-210f783d9c8d`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:10:52,654::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:10:52,654::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:10:52,654::task::990::TaskManager.Task::(_decref) Task=`5df812e9-b3ce-474d-bdcf-210f783d9c8d`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:10:53,861::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:10:54,464::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:54,545::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387685 s, 26.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:10:55,495::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:10:55,503::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00072937 s, 5.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:10:56,558::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:56,638::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384602 s, 26.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:10:56,879::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:10:58,507::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:10:58,507::task::595::TaskManager.Task::(_updateState) Task=`9d03a32d-8322-430d-b5c2-1ddf2601ba0a`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:10:58,508::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:10:58,508::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:10:58,508::task::1185::TaskManager.Task::(prepare) Task=`9d03a32d-8322-430d-b5c2-1ddf2601ba0a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:10:58,508::task::595::TaskManager.Task::(_updateState) Task=`9d03a32d-8322-430d-b5c2-1ddf2601ba0a`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:10:58,508::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:10:58,509::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:10:58,509::task::990::TaskManager.Task::(_decref) Task=`9d03a32d-8322-430d-b5c2-1ddf2601ba0a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:10:58,516::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:10:58,516::task::595::TaskManager.Task::(_updateState) Task=`ded5bafc-0fc6-4003-8bf7-b878c9ce12f5`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:10:58,516::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:10:58,517::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a118c72a-d658-4ecf-bf8a-d35057fc0b67`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:10:58,517::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:10:58,517::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:10:58,517::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a118c72a-d658-4ecf-bf8a-d35057fc0b67`::Granted request >Thread-338::DEBUG::2014-05-13 12:10:58,518::task::827::TaskManager.Task::(resourceAcquired) Task=`ded5bafc-0fc6-4003-8bf7-b878c9ce12f5`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:10:58,518::task::990::TaskManager.Task::(_decref) Task=`ded5bafc-0fc6-4003-8bf7-b878c9ce12f5`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:10:58,518::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:10:58,519::task::1185::TaskManager.Task::(prepare) Task=`ded5bafc-0fc6-4003-8bf7-b878c9ce12f5`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:10:58,519::task::595::TaskManager.Task::(_updateState) Task=`ded5bafc-0fc6-4003-8bf7-b878c9ce12f5`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:10:58,519::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:10:58,519::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:10:58,519::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:10:58,519::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:10:58,520::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:10:58,520::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:10:58,520::task::990::TaskManager.Task::(_decref) Task=`ded5bafc-0fc6-4003-8bf7-b878c9ce12f5`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:10:58,651::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:10:58,727::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0343856 s, 29.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:10:59,900::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:11:00,740::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:11:00,819::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0354054 s, 28.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:11:02,832::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:11:02,915::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0416058 s, 24.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:11:02,918::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:11:04,932::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:11:05,005::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0308005 s, 33.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:11:05,510::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:11:05,518::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000626215 s, 6.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:11:05,938::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:11:07,018::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:11:07,099::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0372595 s, 27.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:11:07,789::task::595::TaskManager.Task::(_updateState) Task=`4733ce41-3364-4803-8851-ba3954cebb2b`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:11:07,789::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:11:07,789::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000626215', 'lastCheck': '2.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:11:07,789::task::1185::TaskManager.Task::(prepare) Task=`4733ce41-3364-4803-8851-ba3954cebb2b`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000626215', 'lastCheck': '2.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:11:07,789::task::595::TaskManager.Task::(_updateState) Task=`4733ce41-3364-4803-8851-ba3954cebb2b`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:11:07,790::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:11:07,790::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:11:07,790::task::990::TaskManager.Task::(_decref) Task=`4733ce41-3364-4803-8851-ba3954cebb2b`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:11:08,540::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:11:08,540::task::595::TaskManager.Task::(_updateState) Task=`9469b7c5-8232-4a21-b5ae-663bd4aab348`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:11:08,541::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:11:08,541::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:11:08,541::task::1185::TaskManager.Task::(prepare) Task=`9469b7c5-8232-4a21-b5ae-663bd4aab348`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:11:08,541::task::595::TaskManager.Task::(_updateState) Task=`9469b7c5-8232-4a21-b5ae-663bd4aab348`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:11:08,542::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:11:08,542::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:11:08,542::task::990::TaskManager.Task::(_decref) Task=`9469b7c5-8232-4a21-b5ae-663bd4aab348`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:11:08,546::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:11:08,547::task::595::TaskManager.Task::(_updateState) Task=`5363358e-3a97-4622-bb04-2ebcaa81b1bf`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:11:08,547::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:11:08,547::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fe08851c-b62a-49cd-9245-cc6ad7558ce9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:11:08,548::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:11:08,548::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:11:08,548::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fe08851c-b62a-49cd-9245-cc6ad7558ce9`::Granted request >Thread-338::DEBUG::2014-05-13 12:11:08,548::task::827::TaskManager.Task::(resourceAcquired) Task=`5363358e-3a97-4622-bb04-2ebcaa81b1bf`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:11:08,549::task::990::TaskManager.Task::(_decref) Task=`5363358e-3a97-4622-bb04-2ebcaa81b1bf`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:11:08,549::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:11:08,550::task::1185::TaskManager.Task::(prepare) Task=`5363358e-3a97-4622-bb04-2ebcaa81b1bf`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:11:08,550::task::595::TaskManager.Task::(_updateState) Task=`5363358e-3a97-4622-bb04-2ebcaa81b1bf`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:11:08,550::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:11:08,550::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:11:08,550::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:11:08,551::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:11:08,551::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:11:08,551::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:11:08,551::task::990::TaskManager.Task::(_decref) Task=`5363358e-3a97-4622-bb04-2ebcaa81b1bf`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:11:09,112::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:11:09,121::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:11:09,191::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0371754 s, 27.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:11:11,204::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:11:11,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317712 s, 32.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:11:12,139::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:11:13,292::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:11:13,372::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365213 s, 28.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:11:15,162::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:11:15,385::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 12:11:15,525::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:11:15,555::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.0152139 s, 269 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:11:15,653::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.224599 s, 4.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:11:17,666::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:11:17,741::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320562 s, 31.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:11:18,181::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:11:18,575::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:11:18,576::task::595::TaskManager.Task::(_updateState) Task=`1e8322d2-cf8f-4274-89b7-22b7f9955406`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:11:18,576::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:11:18,576::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:11:18,577::task::1185::TaskManager.Task::(prepare) Task=`1e8322d2-cf8f-4274-89b7-22b7f9955406`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:11:18,577::task::595::TaskManager.Task::(_updateState) Task=`1e8322d2-cf8f-4274-89b7-22b7f9955406`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:11:18,577::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:11:18,577::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:11:18,577::task::990::TaskManager.Task::(_decref) Task=`1e8322d2-cf8f-4274-89b7-22b7f9955406`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:11:18,582::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:11:18,582::task::595::TaskManager.Task::(_updateState) Task=`46b16d20-af5f-4824-9ce9-acc407615638`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:11:18,582::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:11:18,583::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4d1c023c-458e-4421-bd2d-8cfd7a092452`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:11:18,583::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:11:18,583::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:11:18,583::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4d1c023c-458e-4421-bd2d-8cfd7a092452`::Granted request >Thread-338::DEBUG::2014-05-13 12:11:18,584::task::827::TaskManager.Task::(resourceAcquired) Task=`46b16d20-af5f-4824-9ce9-acc407615638`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:11:18,584::task::990::TaskManager.Task::(_decref) Task=`46b16d20-af5f-4824-9ce9-acc407615638`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:11:18,585::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:11:18,585::task::1185::TaskManager.Task::(prepare) Task=`46b16d20-af5f-4824-9ce9-acc407615638`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:11:18,585::task::595::TaskManager.Task::(_updateState) Task=`46b16d20-af5f-4824-9ce9-acc407615638`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:11:18,585::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:11:18,586::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:11:18,586::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:11:18,586::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:11:18,586::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:11:18,586::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:11:18,587::task::990::TaskManager.Task::(_decref) Task=`46b16d20-af5f-4824-9ce9-acc407615638`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:11:19,754::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:11:19,829::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313785 s, 32.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:11:21,208::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:11:21,842::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:11:21,922::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037722 s, 27.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:11:22,921::task::595::TaskManager.Task::(_updateState) Task=`699d2dd7-2541-48ac-9ee1-7055808d4db9`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:11:22,922::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:11:22,922::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0152139', 'lastCheck': '7.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:11:22,922::task::1185::TaskManager.Task::(prepare) Task=`699d2dd7-2541-48ac-9ee1-7055808d4db9`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0152139', 'lastCheck': '7.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:11:22,922::task::595::TaskManager.Task::(_updateState) Task=`699d2dd7-2541-48ac-9ee1-7055808d4db9`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:11:22,922::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:11:22,923::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:11:22,923::task::990::TaskManager.Task::(_decref) Task=`699d2dd7-2541-48ac-9ee1-7055808d4db9`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:11:23,935::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:11:24,014::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0369052 s, 27.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:11:24,226::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:11:25,584::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:11:25,592::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000643243 s, 6.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:11:26,027::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:11:26,106::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361057 s, 28.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:11:27,244::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:11:28,119::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:11:28,210::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0476533 s, 21.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:11:28,607::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [50b954f9] >Thread-66::DEBUG::2014-05-13 12:11:28,607::task::595::TaskManager.Task::(_updateState) Task=`545f0921-e48a-4a21-8094-acc8c3afa77c`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:11:28,607::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:11:28,608::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:11:28,608::task::1185::TaskManager.Task::(prepare) Task=`545f0921-e48a-4a21-8094-acc8c3afa77c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:11:28,608::task::595::TaskManager.Task::(_updateState) Task=`545f0921-e48a-4a21-8094-acc8c3afa77c`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:11:28,608::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:11:28,608::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:11:28,608::task::990::TaskManager.Task::(_decref) Task=`545f0921-e48a-4a21-8094-acc8c3afa77c`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:11:28,613::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [50b954f9] >Thread-338::DEBUG::2014-05-13 12:11:28,613::task::595::TaskManager.Task::(_updateState) Task=`b52ea550-25b4-4afa-a55a-4735fa9e8035`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:11:28,613::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:11:28,614::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b7c50952-5869-460c-8539-a5757ba6be92`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:11:28,614::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:11:28,614::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:11:28,614::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b7c50952-5869-460c-8539-a5757ba6be92`::Granted request >Thread-338::DEBUG::2014-05-13 12:11:28,615::task::827::TaskManager.Task::(resourceAcquired) Task=`b52ea550-25b4-4afa-a55a-4735fa9e8035`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:11:28,615::task::990::TaskManager.Task::(_decref) Task=`b52ea550-25b4-4afa-a55a-4735fa9e8035`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:11:28,615::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:11:28,616::task::1185::TaskManager.Task::(prepare) Task=`b52ea550-25b4-4afa-a55a-4735fa9e8035`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:11:28,616::task::595::TaskManager.Task::(_updateState) Task=`b52ea550-25b4-4afa-a55a-4735fa9e8035`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:11:28,616::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:11:28,616::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:11:28,616::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:11:28,616::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:11:28,617::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:11:28,617::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:11:28,617::task::990::TaskManager.Task::(_decref) Task=`b52ea550-25b4-4afa-a55a-4735fa9e8035`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:11:30,223::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:11:30,263::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:11:30,299::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.030967 s, 33.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:11:32,312::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:11:32,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0326436 s, 31.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:11:33,280::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:11:34,400::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:11:34,478::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362513 s, 28.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:11:35,599::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:11:35,608::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00173871 s, 2.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:11:36,299::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:11:36,491::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:11:36,571::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355155 s, 28.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:11:38,077::task::595::TaskManager.Task::(_updateState) Task=`337e1258-30cb-40d1-bfa3-765942033190`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:11:38,077::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:11:38,077::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00173871', 'lastCheck': '2.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:11:38,078::task::1185::TaskManager.Task::(prepare) Task=`337e1258-30cb-40d1-bfa3-765942033190`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00173871', 'lastCheck': '2.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:11:38,078::task::595::TaskManager.Task::(_updateState) Task=`337e1258-30cb-40d1-bfa3-765942033190`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:11:38,078::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:11:38,078::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:11:38,078::task::990::TaskManager.Task::(_decref) Task=`337e1258-30cb-40d1-bfa3-765942033190`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:11:38,584::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 12:11:38,636::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5fad7764] >Thread-66::DEBUG::2014-05-13 12:11:38,638::task::595::TaskManager.Task::(_updateState) Task=`3e5cd17a-0300-41b4-b2a4-1b245ab90717`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:11:38,639::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:11:38,641::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:11:38,641::task::1185::TaskManager.Task::(prepare) Task=`3e5cd17a-0300-41b4-b2a4-1b245ab90717`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:11:38,642::task::595::TaskManager.Task::(_updateState) Task=`3e5cd17a-0300-41b4-b2a4-1b245ab90717`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:11:38,643::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:11:38,644::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:11:38,645::task::990::TaskManager.Task::(_decref) Task=`3e5cd17a-0300-41b4-b2a4-1b245ab90717`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:11:38,652::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5fad7764] >Thread-338::DEBUG::2014-05-13 12:11:38,654::task::595::TaskManager.Task::(_updateState) Task=`5a5a649f-68ff-47ff-80d2-7009f94d7410`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:11:38,655::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:11:38,657::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e98024c4-99f0-475d-b88d-7672cd130a39`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:11:38,658::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:11:38,660::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:11:38,661::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e98024c4-99f0-475d-b88d-7672cd130a39`::Granted request >Thread-338::DEBUG::2014-05-13 12:11:38,663::task::827::TaskManager.Task::(resourceAcquired) Task=`5a5a649f-68ff-47ff-80d2-7009f94d7410`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:11:38,664::task::990::TaskManager.Task::(_decref) Task=`5a5a649f-68ff-47ff-80d2-7009f94d7410`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:11:38,668::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:11:38,669::task::1185::TaskManager.Task::(prepare) Task=`5a5a649f-68ff-47ff-80d2-7009f94d7410`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:11:38,670::task::595::TaskManager.Task::(_updateState) Task=`5a5a649f-68ff-47ff-80d2-7009f94d7410`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:11:38,670::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:11:38,671::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:11:38,673::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:11:38,674::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:11:38,675::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:11:38,676::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:11:38,677::task::990::TaskManager.Task::(_decref) Task=`5a5a649f-68ff-47ff-80d2-7009f94d7410`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:11:38,702::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358374 s, 28.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:11:39,317::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:11:40,715::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:11:40,791::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316644 s, 32.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:11:42,345::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:11:42,804::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:11:42,896::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0477712 s, 21.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:11:44,908::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:11:44,983::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317978 s, 32.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:11:45,364::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:11:45,615::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:11:45,623::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000636578 s, 6.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:11:46,996::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:11:47,070::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031614 s, 32.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:11:48,384::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:11:48,698::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:11:48,699::task::595::TaskManager.Task::(_updateState) Task=`338ebc41-fcc5-4a11-822f-9dcbeb818e14`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:11:48,699::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:11:48,699::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:11:48,699::task::1185::TaskManager.Task::(prepare) Task=`338ebc41-fcc5-4a11-822f-9dcbeb818e14`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:11:48,700::task::595::TaskManager.Task::(_updateState) Task=`338ebc41-fcc5-4a11-822f-9dcbeb818e14`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:11:48,700::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:11:48,700::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:11:48,700::task::990::TaskManager.Task::(_decref) Task=`338ebc41-fcc5-4a11-822f-9dcbeb818e14`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:11:48,705::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:11:48,705::task::595::TaskManager.Task::(_updateState) Task=`9f4f0f89-a036-4e6d-a9e4-82314511e69a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:11:48,705::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:11:48,706::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2e241739-5ec7-455c-965d-eac88b4811d8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:11:48,706::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:11:48,706::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:11:48,706::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2e241739-5ec7-455c-965d-eac88b4811d8`::Granted request >Thread-338::DEBUG::2014-05-13 12:11:48,708::task::827::TaskManager.Task::(resourceAcquired) Task=`9f4f0f89-a036-4e6d-a9e4-82314511e69a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:11:48,708::task::990::TaskManager.Task::(_decref) Task=`9f4f0f89-a036-4e6d-a9e4-82314511e69a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:11:48,709::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:11:48,709::task::1185::TaskManager.Task::(prepare) Task=`9f4f0f89-a036-4e6d-a9e4-82314511e69a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:11:48,709::task::595::TaskManager.Task::(_updateState) Task=`9f4f0f89-a036-4e6d-a9e4-82314511e69a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:11:48,709::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:11:48,709::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:11:48,710::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:11:48,710::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:11:48,710::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:11:48,710::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:11:48,711::task::990::TaskManager.Task::(_decref) Task=`9f4f0f89-a036-4e6d-a9e4-82314511e69a`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:11:49,083::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:11:49,163::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361002 s, 28.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:11:51,176::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:11:51,251::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312976 s, 32.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:11:51,410::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:11:53,209::task::595::TaskManager.Task::(_updateState) Task=`b8dd1fb0-1012-4b6e-a027-a8b29e8acf63`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:11:53,209::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:11:53,209::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000636578', 'lastCheck': '7.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:11:53,209::task::1185::TaskManager.Task::(prepare) Task=`b8dd1fb0-1012-4b6e-a027-a8b29e8acf63`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000636578', 'lastCheck': '7.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:11:53,209::task::595::TaskManager.Task::(_updateState) Task=`b8dd1fb0-1012-4b6e-a027-a8b29e8acf63`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:11:53,210::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:11:53,210::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:11:53,210::task::990::TaskManager.Task::(_decref) Task=`b8dd1fb0-1012-4b6e-a027-a8b29e8acf63`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:11:53,264::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:11:53,340::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0324745 s, 31.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:11:54,429::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:11:55,353::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 12:11:55,630::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:11:55,926::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.283865 s, 14.4 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:11:56,002::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.607609 s, 1.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:11:57,448::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:11:58,015::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:11:58,095::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365297 s, 28.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:11:58,732::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:11:58,732::task::595::TaskManager.Task::(_updateState) Task=`6549858a-fdd7-407c-bffb-c83c78ce1818`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:11:58,732::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:11:58,733::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:11:58,733::task::1185::TaskManager.Task::(prepare) Task=`6549858a-fdd7-407c-bffb-c83c78ce1818`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:11:58,733::task::595::TaskManager.Task::(_updateState) Task=`6549858a-fdd7-407c-bffb-c83c78ce1818`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:11:58,733::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:11:58,733::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:11:58,733::task::990::TaskManager.Task::(_decref) Task=`6549858a-fdd7-407c-bffb-c83c78ce1818`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:11:58,738::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:11:58,739::task::595::TaskManager.Task::(_updateState) Task=`33cdd458-0af2-4743-9542-a9312671f8e7`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:11:58,739::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:11:58,739::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`7ddc4edc-8199-4a27-8817-61954997acce`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:11:58,739::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:11:58,740::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:11:58,740::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`7ddc4edc-8199-4a27-8817-61954997acce`::Granted request >Thread-338::DEBUG::2014-05-13 12:11:58,740::task::827::TaskManager.Task::(resourceAcquired) Task=`33cdd458-0af2-4743-9542-a9312671f8e7`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:11:58,740::task::990::TaskManager.Task::(_decref) Task=`33cdd458-0af2-4743-9542-a9312671f8e7`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:11:58,741::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:11:58,741::task::1185::TaskManager.Task::(prepare) Task=`33cdd458-0af2-4743-9542-a9312671f8e7`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:11:58,741::task::595::TaskManager.Task::(_updateState) Task=`33cdd458-0af2-4743-9542-a9312671f8e7`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:11:58,741::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:11:58,742::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:11:58,742::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:11:58,742::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:11:58,742::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:11:58,742::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:11:58,742::task::990::TaskManager.Task::(_decref) Task=`33cdd458-0af2-4743-9542-a9312671f8e7`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:12:00,108::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:00,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313437 s, 32.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:12:00,466::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:12:02,196::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:02,271::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319665 s, 32.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:12:03,493::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:12:04,284::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:04,364::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.03619 s, 28.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:12:05,945::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:12:05,953::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00071469 s, 5.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:12:06,377::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:06,456::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362831 s, 28.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:12:06,511::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:12:08,341::task::595::TaskManager.Task::(_updateState) Task=`4f329a53-9236-4ee1-a528-b027b4fcbd88`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:12:08,342::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:12:08,342::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00071469', 'lastCheck': '2.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:12:08,342::task::1185::TaskManager.Task::(prepare) Task=`4f329a53-9236-4ee1-a528-b027b4fcbd88`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00071469', 'lastCheck': '2.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:12:08,342::task::595::TaskManager.Task::(_updateState) Task=`4f329a53-9236-4ee1-a528-b027b4fcbd88`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:12:08,342::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:12:08,342::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:12:08,343::task::990::TaskManager.Task::(_decref) Task=`4f329a53-9236-4ee1-a528-b027b4fcbd88`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:12:08,469::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 12:12:08,760::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [19876d15] >Thread-66::DEBUG::2014-05-13 12:12:08,761::task::595::TaskManager.Task::(_updateState) Task=`f3fb3cd1-2990-4994-8929-528685093baf`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:12:08,761::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:12:08,762::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:12:08,762::task::1185::TaskManager.Task::(prepare) Task=`f3fb3cd1-2990-4994-8929-528685093baf`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:12:08,762::task::595::TaskManager.Task::(_updateState) Task=`f3fb3cd1-2990-4994-8929-528685093baf`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:12:08,762::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:12:08,762::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:12:08,762::task::990::TaskManager.Task::(_decref) Task=`f3fb3cd1-2990-4994-8929-528685093baf`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:12:08,767::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [19876d15] >Thread-338::DEBUG::2014-05-13 12:12:08,768::task::595::TaskManager.Task::(_updateState) Task=`497a92d9-9f78-4528-9320-be1435406f60`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:12:08,768::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:12:08,768::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`86b97af5-b28f-4a55-86d1-f638a336fe7d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:12:08,769::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:12:08,769::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:12:08,769::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`86b97af5-b28f-4a55-86d1-f638a336fe7d`::Granted request >Thread-338::DEBUG::2014-05-13 12:12:08,771::task::827::TaskManager.Task::(resourceAcquired) Task=`497a92d9-9f78-4528-9320-be1435406f60`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:12:08,772::task::990::TaskManager.Task::(_decref) Task=`497a92d9-9f78-4528-9320-be1435406f60`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:12:08,776::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:12:08,777::task::1185::TaskManager.Task::(prepare) Task=`497a92d9-9f78-4528-9320-be1435406f60`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:12:08,778::task::595::TaskManager.Task::(_updateState) Task=`497a92d9-9f78-4528-9320-be1435406f60`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:12:08,779::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:12:08,780::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:12:08,781::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:12:08,783::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:12:08,783::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:12:08,784::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:12:08,786::task::990::TaskManager.Task::(_decref) Task=`497a92d9-9f78-4528-9320-be1435406f60`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:12:08,849::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.33687 s, 3.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:12:09,535::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:12:10,862::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:10,937::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318765 s, 32.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:12:12,554::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:12:12,950::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:13,031::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384647 s, 26.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:12:15,044::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:15,123::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355099 s, 28.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:12:15,576::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:12:15,960::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:12:15,968::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000717815 s, 5.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:12:17,136::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:17,210::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317611 s, 32.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:12:18,597::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:12:18,805::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:12:18,806::task::595::TaskManager.Task::(_updateState) Task=`5a6fd96c-9e87-46cf-a132-9fc978523816`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:12:18,806::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:12:18,807::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:12:18,807::task::1185::TaskManager.Task::(prepare) Task=`5a6fd96c-9e87-46cf-a132-9fc978523816`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:12:18,807::task::595::TaskManager.Task::(_updateState) Task=`5a6fd96c-9e87-46cf-a132-9fc978523816`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:12:18,807::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:12:18,807::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:12:18,807::task::990::TaskManager.Task::(_decref) Task=`5a6fd96c-9e87-46cf-a132-9fc978523816`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:12:18,812::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:12:18,812::task::595::TaskManager.Task::(_updateState) Task=`8f9438b1-0af9-4731-b852-1177db1a3dc8`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:12:18,812::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:12:18,814::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ffe04b76-a244-47f0-be94-7d2ca89ad13e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:12:18,814::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:12:18,814::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:12:18,814::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ffe04b76-a244-47f0-be94-7d2ca89ad13e`::Granted request >Thread-338::DEBUG::2014-05-13 12:12:18,815::task::827::TaskManager.Task::(resourceAcquired) Task=`8f9438b1-0af9-4731-b852-1177db1a3dc8`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:12:18,815::task::990::TaskManager.Task::(_decref) Task=`8f9438b1-0af9-4731-b852-1177db1a3dc8`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:12:18,816::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:12:18,816::task::1185::TaskManager.Task::(prepare) Task=`8f9438b1-0af9-4731-b852-1177db1a3dc8`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:12:18,816::task::595::TaskManager.Task::(_updateState) Task=`8f9438b1-0af9-4731-b852-1177db1a3dc8`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:12:18,816::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:12:18,817::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:12:18,817::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:12:18,817::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:12:18,817::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:12:18,817::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:12:18,818::task::990::TaskManager.Task::(_decref) Task=`8f9438b1-0af9-4731-b852-1177db1a3dc8`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:12:19,223::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:19,303::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365523 s, 28.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:12:21,316::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:21,391::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0327588 s, 31.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:12:21,616::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:12:23,404::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 12:12:23,471::task::595::TaskManager.Task::(_updateState) Task=`ee6385ce-2e5a-4638-b92f-67cbdbe214eb`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:12:23,471::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:12:23,472::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000717815', 'lastCheck': '7.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:12:23,472::task::1185::TaskManager.Task::(prepare) Task=`ee6385ce-2e5a-4638-b92f-67cbdbe214eb`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000717815', 'lastCheck': '7.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:12:23,472::task::595::TaskManager.Task::(_updateState) Task=`ee6385ce-2e5a-4638-b92f-67cbdbe214eb`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:12:23,472::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:12:23,472::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:12:23,472::task::990::TaskManager.Task::(_decref) Task=`ee6385ce-2e5a-4638-b92f-67cbdbe214eb`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:12:23,775::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.315593 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:12:24,637::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:12:25,788::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:25,862::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031104 s, 32.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:12:25,975::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:12:25,983::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00070779 s, 5.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:12:27,661::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:12:27,875::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:27,956::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0375034 s, 27.3 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:12:28,835::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5affc0d0] >Thread-66::DEBUG::2014-05-13 12:12:28,835::task::595::TaskManager.Task::(_updateState) Task=`b211701a-52c7-4662-852f-a78d7be0105f`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:12:28,836::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:12:28,836::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:12:28,836::task::1185::TaskManager.Task::(prepare) Task=`b211701a-52c7-4662-852f-a78d7be0105f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:12:28,836::task::595::TaskManager.Task::(_updateState) Task=`b211701a-52c7-4662-852f-a78d7be0105f`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:12:28,837::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:12:28,837::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:12:28,837::task::990::TaskManager.Task::(_decref) Task=`b211701a-52c7-4662-852f-a78d7be0105f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:12:28,841::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5affc0d0] >Thread-338::DEBUG::2014-05-13 12:12:28,842::task::595::TaskManager.Task::(_updateState) Task=`1c1fab02-5ff8-4fbb-81b4-248e9df5bd72`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:12:28,842::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:12:28,843::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a94fbd56-0bab-4cf5-980b-5776cb9692d6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:12:28,843::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:12:28,843::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:12:28,843::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a94fbd56-0bab-4cf5-980b-5776cb9692d6`::Granted request >Thread-338::DEBUG::2014-05-13 12:12:28,844::task::827::TaskManager.Task::(resourceAcquired) Task=`1c1fab02-5ff8-4fbb-81b4-248e9df5bd72`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:12:28,844::task::990::TaskManager.Task::(_decref) Task=`1c1fab02-5ff8-4fbb-81b4-248e9df5bd72`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:12:28,845::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:12:28,845::task::1185::TaskManager.Task::(prepare) Task=`1c1fab02-5ff8-4fbb-81b4-248e9df5bd72`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:12:28,845::task::595::TaskManager.Task::(_updateState) Task=`1c1fab02-5ff8-4fbb-81b4-248e9df5bd72`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:12:28,845::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:12:28,845::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:12:28,846::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:12:28,846::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:12:28,846::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:12:28,846::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:12:28,847::task::990::TaskManager.Task::(_decref) Task=`1c1fab02-5ff8-4fbb-81b4-248e9df5bd72`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:12:29,970::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:30,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353937 s, 28.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:12:30,681::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:12:32,062::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:32,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311066 s, 32.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:12:33,699::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:12:34,147::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:34,226::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355838 s, 28.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:12:35,990::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:12:35,997::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000595769 s, 6.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:12:36,239::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:36,609::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.327887 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:12:36,720::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:12:38,607::task::595::TaskManager.Task::(_updateState) Task=`71773172-81b9-42c1-a710-675d842178f9`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:12:38,607::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:12:38,607::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000595769', 'lastCheck': '2.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:12:38,607::task::1185::TaskManager.Task::(prepare) Task=`71773172-81b9-42c1-a710-675d842178f9`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000595769', 'lastCheck': '2.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:12:38,608::task::595::TaskManager.Task::(_updateState) Task=`71773172-81b9-42c1-a710-675d842178f9`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:12:38,608::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:12:38,608::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:12:38,608::task::990::TaskManager.Task::(_decref) Task=`71773172-81b9-42c1-a710-675d842178f9`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:12:38,622::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:38,700::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360042 s, 28.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:12:38,867::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:12:38,867::task::595::TaskManager.Task::(_updateState) Task=`0d8fa8a8-a198-4cc1-a1e4-eeb20e2ed3d2`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:12:38,867::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:12:38,868::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:12:38,868::task::1185::TaskManager.Task::(prepare) Task=`0d8fa8a8-a198-4cc1-a1e4-eeb20e2ed3d2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:12:38,868::task::595::TaskManager.Task::(_updateState) Task=`0d8fa8a8-a198-4cc1-a1e4-eeb20e2ed3d2`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:12:38,868::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:12:38,868::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:12:38,869::task::990::TaskManager.Task::(_decref) Task=`0d8fa8a8-a198-4cc1-a1e4-eeb20e2ed3d2`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:12:38,873::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:12:38,873::task::595::TaskManager.Task::(_updateState) Task=`89c8bcb5-bba3-4b55-9bff-27ae2f05ccf4`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:12:38,873::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:12:38,874::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`cd5e2c91-92fb-4cc6-a486-7cc1c2456de7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:12:38,874::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:12:38,874::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:12:38,875::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`cd5e2c91-92fb-4cc6-a486-7cc1c2456de7`::Granted request >Thread-338::DEBUG::2014-05-13 12:12:38,875::task::827::TaskManager.Task::(resourceAcquired) Task=`89c8bcb5-bba3-4b55-9bff-27ae2f05ccf4`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:12:38,875::task::990::TaskManager.Task::(_decref) Task=`89c8bcb5-bba3-4b55-9bff-27ae2f05ccf4`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:12:38,876::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:12:38,876::task::1185::TaskManager.Task::(prepare) Task=`89c8bcb5-bba3-4b55-9bff-27ae2f05ccf4`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:12:38,876::task::595::TaskManager.Task::(_updateState) Task=`89c8bcb5-bba3-4b55-9bff-27ae2f05ccf4`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:12:38,876::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:12:38,877::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:12:38,877::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:12:38,877::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:12:38,877::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:12:38,877::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:12:38,878::task::990::TaskManager.Task::(_decref) Task=`89c8bcb5-bba3-4b55-9bff-27ae2f05ccf4`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:12:39,738::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:12:40,713::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:40,788::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314826 s, 32.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:12:42,757::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:12:42,801::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:42,880::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355682 s, 28.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:12:44,893::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:44,972::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359383 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:12:45,777::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:12:46,005::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:12:46,013::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000753156 s, 5.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:12:46,985::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:47,060::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.03125 s, 32.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:12:48,796::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:12:48,896::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:12:48,896::task::595::TaskManager.Task::(_updateState) Task=`3b53390b-3df6-45b5-9796-929d3797a197`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:12:48,896::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:12:48,897::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:12:48,897::task::1185::TaskManager.Task::(prepare) Task=`3b53390b-3df6-45b5-9796-929d3797a197`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:12:48,897::task::595::TaskManager.Task::(_updateState) Task=`3b53390b-3df6-45b5-9796-929d3797a197`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:12:48,897::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:12:48,897::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:12:48,898::task::990::TaskManager.Task::(_decref) Task=`3b53390b-3df6-45b5-9796-929d3797a197`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:12:49,073::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:49,165::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0482943 s, 21.2 MB/s\n'; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:12:49,423::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:12:49,423::task::595::TaskManager.Task::(_updateState) Task=`fb05f638-f27b-4b06-b8cc-39e5cebf5290`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:12:49,423::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:12:49,424::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a4f58b11-a222-483d-84c9-3ed0f9cdcfcc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:12:49,424::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:12:49,424::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:12:49,424::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a4f58b11-a222-483d-84c9-3ed0f9cdcfcc`::Granted request >Thread-338::DEBUG::2014-05-13 12:12:49,425::task::827::TaskManager.Task::(resourceAcquired) Task=`fb05f638-f27b-4b06-b8cc-39e5cebf5290`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:12:49,425::task::990::TaskManager.Task::(_decref) Task=`fb05f638-f27b-4b06-b8cc-39e5cebf5290`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:12:49,426::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:12:49,426::task::1185::TaskManager.Task::(prepare) Task=`fb05f638-f27b-4b06-b8cc-39e5cebf5290`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:12:49,426::task::595::TaskManager.Task::(_updateState) Task=`fb05f638-f27b-4b06-b8cc-39e5cebf5290`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:12:49,426::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:12:49,427::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:12:49,427::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:12:49,427::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:12:49,427::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:12:49,427::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:12:49,428::task::990::TaskManager.Task::(_decref) Task=`fb05f638-f27b-4b06-b8cc-39e5cebf5290`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:12:51,178::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:51,254::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0340456 s, 30.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:12:51,821::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:12:53,267::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:53,342::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320688 s, 31.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:12:53,753::task::595::TaskManager.Task::(_updateState) Task=`76a3d80e-538b-47c3-a850-17de6e1b41dd`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:12:53,753::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:12:53,753::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000753156', 'lastCheck': '7.7', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:12:53,754::task::1185::TaskManager.Task::(prepare) Task=`76a3d80e-538b-47c3-a850-17de6e1b41dd`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000753156', 'lastCheck': '7.7', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:12:53,754::task::595::TaskManager.Task::(_updateState) Task=`76a3d80e-538b-47c3-a850-17de6e1b41dd`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:12:53,754::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:12:53,754::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:12:53,754::task::990::TaskManager.Task::(_decref) Task=`76a3d80e-538b-47c3-a850-17de6e1b41dd`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:12:54,839::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:12:55,355::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:55,436::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374305 s, 27.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:12:56,020::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:12:56,027::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000700514 s, 5.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:12:57,450::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:57,531::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0377398 s, 27.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:12:57,856::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:12:59,444::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:12:59,445::task::595::TaskManager.Task::(_updateState) Task=`ba2425f2-eb37-442e-a5d1-081ca9df9a38`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:12:59,445::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:12:59,446::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:12:59,446::task::1185::TaskManager.Task::(prepare) Task=`ba2425f2-eb37-442e-a5d1-081ca9df9a38`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:12:59,446::task::595::TaskManager.Task::(_updateState) Task=`ba2425f2-eb37-442e-a5d1-081ca9df9a38`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:12:59,446::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:12:59,446::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:12:59,447::task::990::TaskManager.Task::(_decref) Task=`ba2425f2-eb37-442e-a5d1-081ca9df9a38`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:12:59,452::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:12:59,452::task::595::TaskManager.Task::(_updateState) Task=`b6ff617c-4438-43de-b015-811357cd6480`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:12:59,452::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:12:59,453::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c4e3f8f5-4344-4fdd-b6ce-0943c4e48e2a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:12:59,453::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:12:59,453::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:12:59,453::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c4e3f8f5-4344-4fdd-b6ce-0943c4e48e2a`::Granted request >Thread-338::DEBUG::2014-05-13 12:12:59,454::task::827::TaskManager.Task::(resourceAcquired) Task=`b6ff617c-4438-43de-b015-811357cd6480`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:12:59,454::task::990::TaskManager.Task::(_decref) Task=`b6ff617c-4438-43de-b015-811357cd6480`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:12:59,455::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:12:59,455::task::1185::TaskManager.Task::(prepare) Task=`b6ff617c-4438-43de-b015-811357cd6480`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:12:59,455::task::595::TaskManager.Task::(_updateState) Task=`b6ff617c-4438-43de-b015-811357cd6480`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:12:59,455::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:12:59,456::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:12:59,456::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:12:59,456::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:12:59,456::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:12:59,456::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:12:59,457::task::990::TaskManager.Task::(_decref) Task=`b6ff617c-4438-43de-b015-811357cd6480`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:12:59,544::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:12:59,622::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035898 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:13:00,873::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:13:01,635::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:01,710::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317974 s, 32.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:13:03,723::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:03,812::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0464051 s, 22.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:13:03,890::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:13:05,825::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:05,913::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305943 s, 33.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:13:06,063::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:13:06,071::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000703013 s, 5.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:13:06,916::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:13:07,927::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:08,003::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031798 s, 32.2 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:13:08,927::task::595::TaskManager.Task::(_updateState) Task=`4068b868-e2d7-4444-9164-94ef49270614`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:13:08,927::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:13:08,927::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000703013', 'lastCheck': '2.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:13:08,928::task::1185::TaskManager.Task::(prepare) Task=`4068b868-e2d7-4444-9164-94ef49270614`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000703013', 'lastCheck': '2.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:13:08,928::task::595::TaskManager.Task::(_updateState) Task=`4068b868-e2d7-4444-9164-94ef49270614`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:13:08,928::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:13:08,928::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:13:08,928::task::990::TaskManager.Task::(_decref) Task=`4068b868-e2d7-4444-9164-94ef49270614`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:13:09,474::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:13:09,474::task::595::TaskManager.Task::(_updateState) Task=`b48cc477-50ba-43c2-873c-e40848e88f76`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:13:09,474::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:13:09,474::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:13:09,475::task::1185::TaskManager.Task::(prepare) Task=`b48cc477-50ba-43c2-873c-e40848e88f76`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:13:09,475::task::595::TaskManager.Task::(_updateState) Task=`b48cc477-50ba-43c2-873c-e40848e88f76`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:13:09,475::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:13:09,475::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:13:09,475::task::990::TaskManager.Task::(_decref) Task=`b48cc477-50ba-43c2-873c-e40848e88f76`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:13:09,480::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:13:09,480::task::595::TaskManager.Task::(_updateState) Task=`74584a89-b27a-4767-be07-fb3118ed4396`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:13:09,480::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:13:09,481::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`39567656-2b70-4deb-8ff9-d411319e4444`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:13:09,481::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:13:09,481::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:13:09,481::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`39567656-2b70-4deb-8ff9-d411319e4444`::Granted request >Thread-338::DEBUG::2014-05-13 12:13:09,482::task::827::TaskManager.Task::(resourceAcquired) Task=`74584a89-b27a-4767-be07-fb3118ed4396`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:13:09,482::task::990::TaskManager.Task::(_decref) Task=`74584a89-b27a-4767-be07-fb3118ed4396`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:13:09,483::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:13:09,483::task::1185::TaskManager.Task::(prepare) Task=`74584a89-b27a-4767-be07-fb3118ed4396`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:13:09,483::task::595::TaskManager.Task::(_updateState) Task=`74584a89-b27a-4767-be07-fb3118ed4396`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:13:09,483::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:13:09,483::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:13:09,484::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:13:09,484::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:13:09,484::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:13:09,484::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:13:09,484::task::990::TaskManager.Task::(_decref) Task=`74584a89-b27a-4767-be07-fb3118ed4396`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:13:09,938::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:13:10,016::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:10,098::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383255 s, 26.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:13:12,111::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:12,184::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318707 s, 32.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:13:12,957::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:13:14,197::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:14,276::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0354955 s, 28.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:13:15,975::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:13:16,078::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:13:16,086::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000653034 s, 6.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:13:16,289::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:16,379::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0469322 s, 21.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:13:18,392::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:18,466::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0333168 s, 30.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:13:18,993::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:13:19,503::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:13:19,504::task::595::TaskManager.Task::(_updateState) Task=`531eaaca-824f-443d-afc1-d15cad27c11d`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:13:19,504::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:13:19,504::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:13:19,504::task::1185::TaskManager.Task::(prepare) Task=`531eaaca-824f-443d-afc1-d15cad27c11d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:13:19,505::task::595::TaskManager.Task::(_updateState) Task=`531eaaca-824f-443d-afc1-d15cad27c11d`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:13:19,505::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:13:19,505::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:13:19,505::task::990::TaskManager.Task::(_decref) Task=`531eaaca-824f-443d-afc1-d15cad27c11d`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:13:19,511::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:13:19,511::task::595::TaskManager.Task::(_updateState) Task=`af2f480a-0d4f-47bc-94af-6b70a0b3e013`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:13:19,511::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:13:19,512::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`715c13d0-3185-42e3-b40b-53b75521399a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:13:19,512::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:13:19,512::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:13:19,513::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`715c13d0-3185-42e3-b40b-53b75521399a`::Granted request >Thread-338::DEBUG::2014-05-13 12:13:19,513::task::827::TaskManager.Task::(resourceAcquired) Task=`af2f480a-0d4f-47bc-94af-6b70a0b3e013`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:13:19,513::task::990::TaskManager.Task::(_decref) Task=`af2f480a-0d4f-47bc-94af-6b70a0b3e013`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:13:19,514::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:13:19,514::task::1185::TaskManager.Task::(prepare) Task=`af2f480a-0d4f-47bc-94af-6b70a0b3e013`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:13:19,514::task::595::TaskManager.Task::(_updateState) Task=`af2f480a-0d4f-47bc-94af-6b70a0b3e013`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:13:19,514::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:13:19,515::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:13:19,515::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:13:19,515::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:13:19,515::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:13:19,515::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:13:19,516::task::990::TaskManager.Task::(_decref) Task=`af2f480a-0d4f-47bc-94af-6b70a0b3e013`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:13:20,479::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:20,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.032348 s, 31.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:13:22,012::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:13:22,566::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:22,641::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0324434 s, 31.6 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:13:24,063::task::595::TaskManager.Task::(_updateState) Task=`b3d96ada-7b2d-4e29-9b2f-46b49f1cd7fb`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:13:24,064::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:13:24,064::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000653034', 'lastCheck': '8.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:13:24,064::task::1185::TaskManager.Task::(prepare) Task=`b3d96ada-7b2d-4e29-9b2f-46b49f1cd7fb`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000653034', 'lastCheck': '8.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:13:24,064::task::595::TaskManager.Task::(_updateState) Task=`b3d96ada-7b2d-4e29-9b2f-46b49f1cd7fb`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:13:24,064::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:13:24,064::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:13:24,065::task::990::TaskManager.Task::(_decref) Task=`b3d96ada-7b2d-4e29-9b2f-46b49f1cd7fb`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:13:24,654::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:24,735::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036616 s, 28.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:13:25,029::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:13:26,093::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:13:26,101::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000667945 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:13:26,748::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:26,828::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382329 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:13:28,053::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:13:28,842::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:29,254::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.369586 s, 2.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:13:29,536::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4a63f6c8] >Thread-66::DEBUG::2014-05-13 12:13:29,536::task::595::TaskManager.Task::(_updateState) Task=`4a6b1b7f-821e-4ac2-b0d4-b4143ee85585`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:13:29,537::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:13:29,537::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:13:29,537::task::1185::TaskManager.Task::(prepare) Task=`4a6b1b7f-821e-4ac2-b0d4-b4143ee85585`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:13:29,537::task::595::TaskManager.Task::(_updateState) Task=`4a6b1b7f-821e-4ac2-b0d4-b4143ee85585`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:13:29,537::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:13:29,538::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:13:29,538::task::990::TaskManager.Task::(_decref) Task=`4a6b1b7f-821e-4ac2-b0d4-b4143ee85585`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:13:29,543::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4a63f6c8] >Thread-338::DEBUG::2014-05-13 12:13:29,544::task::595::TaskManager.Task::(_updateState) Task=`670642a2-251e-4807-9813-67f1b41b5dfe`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:13:29,544::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:13:29,544::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0a9b6ac4-5a48-4898-9fcf-120fc88122b4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:13:29,545::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:13:29,546::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:13:29,546::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0a9b6ac4-5a48-4898-9fcf-120fc88122b4`::Granted request >Thread-338::DEBUG::2014-05-13 12:13:29,546::task::827::TaskManager.Task::(resourceAcquired) Task=`670642a2-251e-4807-9813-67f1b41b5dfe`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:13:29,546::task::990::TaskManager.Task::(_decref) Task=`670642a2-251e-4807-9813-67f1b41b5dfe`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:13:29,547::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:13:29,547::task::1185::TaskManager.Task::(prepare) Task=`670642a2-251e-4807-9813-67f1b41b5dfe`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:13:29,548::task::595::TaskManager.Task::(_updateState) Task=`670642a2-251e-4807-9813-67f1b41b5dfe`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:13:29,548::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:13:29,548::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:13:29,548::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:13:29,548::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:13:29,548::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:13:29,549::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:13:29,549::task::990::TaskManager.Task::(_decref) Task=`670642a2-251e-4807-9813-67f1b41b5dfe`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:13:31,073::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:13:31,267::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:31,346::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356295 s, 28.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:13:33,359::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:33,435::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319841 s, 32.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:13:34,092::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:13:35,448::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:35,523::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315558 s, 32.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:13:36,108::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:13:36,116::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00060868 s, 6.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:13:37,111::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:13:37,536::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:37,610::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311747 s, 32.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:13:39,202::task::595::TaskManager.Task::(_updateState) Task=`6020325c-9508-4882-8344-1f605bf05a13`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:13:39,202::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:13:39,202::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00060868', 'lastCheck': '3.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:13:39,202::task::1185::TaskManager.Task::(prepare) Task=`6020325c-9508-4882-8344-1f605bf05a13`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00060868', 'lastCheck': '3.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:13:39,203::task::595::TaskManager.Task::(_updateState) Task=`6020325c-9508-4882-8344-1f605bf05a13`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:13:39,203::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:13:39,203::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:13:39,203::task::990::TaskManager.Task::(_decref) Task=`6020325c-9508-4882-8344-1f605bf05a13`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:13:39,574::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:13:39,575::task::595::TaskManager.Task::(_updateState) Task=`3a0473ce-3708-4b2a-90f7-5ba18619d6c7`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:13:39,575::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:13:39,575::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:13:39,575::task::1185::TaskManager.Task::(prepare) Task=`3a0473ce-3708-4b2a-90f7-5ba18619d6c7`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:13:39,575::task::595::TaskManager.Task::(_updateState) Task=`3a0473ce-3708-4b2a-90f7-5ba18619d6c7`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:13:39,576::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:13:39,576::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:13:39,576::task::990::TaskManager.Task::(_decref) Task=`3a0473ce-3708-4b2a-90f7-5ba18619d6c7`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:13:39,581::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:13:39,581::task::595::TaskManager.Task::(_updateState) Task=`f8806352-a498-40ad-ac8d-421a4b44b0e0`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:13:39,582::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:13:39,582::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`60aff938-4873-471e-838b-178dadc9962c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:13:39,582::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:13:39,583::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:13:39,583::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`60aff938-4873-471e-838b-178dadc9962c`::Granted request >Thread-338::DEBUG::2014-05-13 12:13:39,583::task::827::TaskManager.Task::(resourceAcquired) Task=`f8806352-a498-40ad-ac8d-421a4b44b0e0`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:13:39,583::task::990::TaskManager.Task::(_decref) Task=`f8806352-a498-40ad-ac8d-421a4b44b0e0`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:13:39,584::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:13:39,584::task::1185::TaskManager.Task::(prepare) Task=`f8806352-a498-40ad-ac8d-421a4b44b0e0`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:13:39,584::task::595::TaskManager.Task::(_updateState) Task=`f8806352-a498-40ad-ac8d-421a4b44b0e0`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:13:39,585::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:13:39,585::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:13:39,585::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:13:39,585::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:13:39,585::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:13:39,585::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:13:39,586::task::990::TaskManager.Task::(_decref) Task=`f8806352-a498-40ad-ac8d-421a4b44b0e0`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:13:39,623::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:39,703::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365409 s, 28.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:13:40,132::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:13:41,716::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:42,115::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.356726 s, 2.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:13:43,152::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:13:44,128::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:44,207::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361139 s, 28.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:13:46,123::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:13:46,130::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000634953 s, 6.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:13:46,169::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:13:46,220::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:46,298::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358134 s, 28.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:13:48,311::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:48,384::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316612 s, 32.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:13:49,194::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:13:49,605::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:13:49,606::task::595::TaskManager.Task::(_updateState) Task=`43c777a1-2691-4272-a39a-9b478d4f09de`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:13:49,606::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:13:49,606::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:13:49,606::task::1185::TaskManager.Task::(prepare) Task=`43c777a1-2691-4272-a39a-9b478d4f09de`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:13:49,607::task::595::TaskManager.Task::(_updateState) Task=`43c777a1-2691-4272-a39a-9b478d4f09de`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:13:49,607::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:13:49,607::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:13:49,607::task::990::TaskManager.Task::(_decref) Task=`43c777a1-2691-4272-a39a-9b478d4f09de`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:13:49,612::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:13:49,612::task::595::TaskManager.Task::(_updateState) Task=`d42c9717-850d-47dc-aa30-836366ddd9e7`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:13:49,613::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:13:49,613::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d6d7ba67-826b-48cc-940c-0c1adfb9a2d8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:13:49,613::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:13:49,613::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:13:49,614::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d6d7ba67-826b-48cc-940c-0c1adfb9a2d8`::Granted request >Thread-338::DEBUG::2014-05-13 12:13:49,614::task::827::TaskManager.Task::(resourceAcquired) Task=`d42c9717-850d-47dc-aa30-836366ddd9e7`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:13:49,614::task::990::TaskManager.Task::(_decref) Task=`d42c9717-850d-47dc-aa30-836366ddd9e7`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:13:49,615::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:13:49,615::task::1185::TaskManager.Task::(prepare) Task=`d42c9717-850d-47dc-aa30-836366ddd9e7`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:13:49,615::task::595::TaskManager.Task::(_updateState) Task=`d42c9717-850d-47dc-aa30-836366ddd9e7`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:13:49,615::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:13:49,615::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:13:49,616::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:13:49,616::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:13:49,616::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:13:49,616::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:13:49,616::task::990::TaskManager.Task::(_decref) Task=`d42c9717-850d-47dc-aa30-836366ddd9e7`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:13:50,397::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:50,471::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310058 s, 33.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:13:52,220::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:13:52,484::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:52,563::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359472 s, 28.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:13:54,353::task::595::TaskManager.Task::(_updateState) Task=`23fde239-6fd3-40dc-967e-9e4c54f69e18`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:13:54,353::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:13:54,353::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000634953', 'lastCheck': '8.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:13:54,353::task::1185::TaskManager.Task::(prepare) Task=`23fde239-6fd3-40dc-967e-9e4c54f69e18`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000634953', 'lastCheck': '8.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:13:54,354::task::595::TaskManager.Task::(_updateState) Task=`23fde239-6fd3-40dc-967e-9e4c54f69e18`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:13:54,354::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:13:54,354::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:13:54,354::task::990::TaskManager.Task::(_decref) Task=`23fde239-6fd3-40dc-967e-9e4c54f69e18`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:13:54,575::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:54,943::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.325752 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:13:55,240::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:13:56,137::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:13:56,145::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000586487 s, 7.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:13:56,956::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:57,035::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358239 s, 28.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:13:58,259::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:13:59,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:13:59,125::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356152 s, 28.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:13:59,635::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:13:59,635::task::595::TaskManager.Task::(_updateState) Task=`156ee458-2f7c-4ed8-b732-d6b2156edc6c`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:13:59,635::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:13:59,636::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:13:59,636::task::1185::TaskManager.Task::(prepare) Task=`156ee458-2f7c-4ed8-b732-d6b2156edc6c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:13:59,636::task::595::TaskManager.Task::(_updateState) Task=`156ee458-2f7c-4ed8-b732-d6b2156edc6c`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:13:59,636::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:13:59,636::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:13:59,637::task::990::TaskManager.Task::(_decref) Task=`156ee458-2f7c-4ed8-b732-d6b2156edc6c`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:13:59,642::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:13:59,642::task::595::TaskManager.Task::(_updateState) Task=`b949c2a0-45d3-4543-b681-63651025fc19`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:13:59,642::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:13:59,643::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a86f4c34-ee1e-4146-9e76-32409d67b212`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:13:59,643::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:13:59,643::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:13:59,644::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a86f4c34-ee1e-4146-9e76-32409d67b212`::Granted request >Thread-338::DEBUG::2014-05-13 12:13:59,644::task::827::TaskManager.Task::(resourceAcquired) Task=`b949c2a0-45d3-4543-b681-63651025fc19`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:13:59,644::task::990::TaskManager.Task::(_decref) Task=`b949c2a0-45d3-4543-b681-63651025fc19`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:13:59,645::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:13:59,645::task::1185::TaskManager.Task::(prepare) Task=`b949c2a0-45d3-4543-b681-63651025fc19`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:13:59,645::task::595::TaskManager.Task::(_updateState) Task=`b949c2a0-45d3-4543-b681-63651025fc19`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:13:59,645::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:13:59,646::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:13:59,646::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:13:59,646::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:13:59,646::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:13:59,646::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:13:59,647::task::990::TaskManager.Task::(_decref) Task=`b949c2a0-45d3-4543-b681-63651025fc19`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:14:01,138::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:14:01,219::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373594 s, 27.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:14:01,275::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:14:03,232::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:14:03,307::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0331257 s, 30.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:14:04,292::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:14:05,320::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:14:05,394::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323214 s, 31.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:14:06,152::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:14:06,160::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000614635 s, 6.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:14:07,311::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:14:07,408::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:14:07,486::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358252 s, 28.6 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:14:09,494::task::595::TaskManager.Task::(_updateState) Task=`9063eaef-3ec7-435e-80d6-a765ae6d25c5`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:14:09,495::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:14:09,495::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000614635', 'lastCheck': '3.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:14:09,495::task::1185::TaskManager.Task::(prepare) Task=`9063eaef-3ec7-435e-80d6-a765ae6d25c5`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000614635', 'lastCheck': '3.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:14:09,495::task::595::TaskManager.Task::(_updateState) Task=`9063eaef-3ec7-435e-80d6-a765ae6d25c5`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:14:09,495::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:14:09,496::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:14:09,496::task::990::TaskManager.Task::(_decref) Task=`9063eaef-3ec7-435e-80d6-a765ae6d25c5`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:14:09,499::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 12:14:09,665::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [45b9a91f] >Thread-66::DEBUG::2014-05-13 12:14:09,666::task::595::TaskManager.Task::(_updateState) Task=`b8a5b050-83e8-4d80-8405-3d94cc9ae5ef`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:14:09,667::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:14:09,667::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:14:09,667::task::1185::TaskManager.Task::(prepare) Task=`b8a5b050-83e8-4d80-8405-3d94cc9ae5ef`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:14:09,667::task::595::TaskManager.Task::(_updateState) Task=`b8a5b050-83e8-4d80-8405-3d94cc9ae5ef`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:14:09,667::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:14:09,668::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:14:09,668::task::990::TaskManager.Task::(_decref) Task=`b8a5b050-83e8-4d80-8405-3d94cc9ae5ef`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:14:09,673::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [45b9a91f] >Thread-338::DEBUG::2014-05-13 12:14:09,673::task::595::TaskManager.Task::(_updateState) Task=`59f529ff-949a-47c3-951e-b337dc255fc5`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:14:09,673::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:14:09,674::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2fdb34ab-2c67-4bac-9b1c-598ffb11e434`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:14:09,674::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:14:09,674::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:14:09,675::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2fdb34ab-2c67-4bac-9b1c-598ffb11e434`::Granted request >Thread-338::DEBUG::2014-05-13 12:14:09,675::task::827::TaskManager.Task::(resourceAcquired) Task=`59f529ff-949a-47c3-951e-b337dc255fc5`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:14:09,675::task::990::TaskManager.Task::(_decref) Task=`59f529ff-949a-47c3-951e-b337dc255fc5`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:14:09,676::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:14:09,676::task::1185::TaskManager.Task::(prepare) Task=`59f529ff-949a-47c3-951e-b337dc255fc5`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:14:09,676::task::595::TaskManager.Task::(_updateState) Task=`59f529ff-949a-47c3-951e-b337dc255fc5`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:14:09,676::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:14:09,677::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:14:09,677::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:14:09,677::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:14:09,677::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:14:09,677::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:14:09,678::task::990::TaskManager.Task::(_decref) Task=`59f529ff-949a-47c3-951e-b337dc255fc5`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:14:09,875::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.332761 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:14:10,336::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:14:11,888::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:14:11,961::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305161 s, 33.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:14:13,357::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:14:13,974::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:14:14,067::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0370185 s, 27.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:14:16,079::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:14:16,162::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0385895 s, 26.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:14:16,181::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:14:16,197::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000659295 s, 6.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:14:16,375::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:14:18,220::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:14:18,296::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321258 s, 31.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:14:19,393::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:14:19,698::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4478bd69] >Thread-66::DEBUG::2014-05-13 12:14:19,698::task::595::TaskManager.Task::(_updateState) Task=`bfcc35ac-e570-4aa8-b2a0-d2452401becb`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:14:19,698::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:14:19,699::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:14:19,699::task::1185::TaskManager.Task::(prepare) Task=`bfcc35ac-e570-4aa8-b2a0-d2452401becb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:14:19,699::task::595::TaskManager.Task::(_updateState) Task=`bfcc35ac-e570-4aa8-b2a0-d2452401becb`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:14:19,699::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:14:19,699::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:14:19,699::task::990::TaskManager.Task::(_decref) Task=`bfcc35ac-e570-4aa8-b2a0-d2452401becb`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:14:19,704::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4478bd69] >Thread-338::DEBUG::2014-05-13 12:14:19,704::task::595::TaskManager.Task::(_updateState) Task=`b8d88bc1-7bfd-4562-b05e-db1f9914df71`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:14:19,705::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:14:19,705::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d9cce5eb-b2f1-4876-9924-a6f1c302ce45`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:14:19,705::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:14:19,705::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:14:19,706::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d9cce5eb-b2f1-4876-9924-a6f1c302ce45`::Granted request >Thread-338::DEBUG::2014-05-13 12:14:19,706::task::827::TaskManager.Task::(resourceAcquired) Task=`b8d88bc1-7bfd-4562-b05e-db1f9914df71`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:14:19,706::task::990::TaskManager.Task::(_decref) Task=`b8d88bc1-7bfd-4562-b05e-db1f9914df71`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:14:19,707::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:14:19,707::task::1185::TaskManager.Task::(prepare) Task=`b8d88bc1-7bfd-4562-b05e-db1f9914df71`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:14:19,707::task::595::TaskManager.Task::(_updateState) Task=`b8d88bc1-7bfd-4562-b05e-db1f9914df71`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:14:19,707::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:14:19,707::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:14:19,707::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:14:19,708::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:14:19,708::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:14:19,708::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:14:19,708::task::990::TaskManager.Task::(_decref) Task=`b8d88bc1-7bfd-4562-b05e-db1f9914df71`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:14:20,309::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:14:20,397::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0455779 s, 22.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:14:22,410::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:14:22,417::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:14:22,493::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387376 s, 26.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:14:24,506::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:14:24,585::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353352 s, 29.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:14:24,632::task::595::TaskManager.Task::(_updateState) Task=`eb50bf70-2bc7-4ecd-a138-9a18a934e505`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:14:24,633::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:14:24,633::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000659295', 'lastCheck': '8.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:14:24,633::task::1185::TaskManager.Task::(prepare) Task=`eb50bf70-2bc7-4ecd-a138-9a18a934e505`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000659295', 'lastCheck': '8.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:14:24,633::task::595::TaskManager.Task::(_updateState) Task=`eb50bf70-2bc7-4ecd-a138-9a18a934e505`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:14:24,633::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:14:24,634::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:14:24,634::task::990::TaskManager.Task::(_decref) Task=`eb50bf70-2bc7-4ecd-a138-9a18a934e505`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:14:25,433::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:14:26,215::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:14:26,223::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000604556 s, 6.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:14:26,598::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:14:26,677::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374065 s, 27.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:14:28,453::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:14:28,690::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:14:28,764::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306984 s, 33.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:14:29,727::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:14:29,728::task::595::TaskManager.Task::(_updateState) Task=`5e0050c3-8d42-48dc-8769-068ca831f5f1`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:14:29,728::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:14:29,729::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:14:29,729::task::1185::TaskManager.Task::(prepare) Task=`5e0050c3-8d42-48dc-8769-068ca831f5f1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:14:29,729::task::595::TaskManager.Task::(_updateState) Task=`5e0050c3-8d42-48dc-8769-068ca831f5f1`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:14:29,729::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:14:29,729::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:14:29,729::task::990::TaskManager.Task::(_decref) Task=`5e0050c3-8d42-48dc-8769-068ca831f5f1`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:14:29,734::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:14:29,734::task::595::TaskManager.Task::(_updateState) Task=`eb744e90-3b13-41b0-bd27-ea8c368f9e28`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:14:29,734::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:14:29,735::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e155700b-0da6-42a3-91e2-979120927b67`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:14:29,735::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:14:29,735::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:14:29,735::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e155700b-0da6-42a3-91e2-979120927b67`::Granted request >Thread-338::DEBUG::2014-05-13 12:14:29,736::task::827::TaskManager.Task::(resourceAcquired) Task=`eb744e90-3b13-41b0-bd27-ea8c368f9e28`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:14:29,736::task::990::TaskManager.Task::(_decref) Task=`eb744e90-3b13-41b0-bd27-ea8c368f9e28`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:14:29,737::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:14:29,737::task::1185::TaskManager.Task::(prepare) Task=`eb744e90-3b13-41b0-bd27-ea8c368f9e28`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:14:29,737::task::595::TaskManager.Task::(_updateState) Task=`eb744e90-3b13-41b0-bd27-ea8c368f9e28`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:14:29,737::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:14:29,738::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:14:29,738::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:14:29,738::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:14:29,738::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:14:29,738::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:14:29,739::task::990::TaskManager.Task::(_decref) Task=`eb744e90-3b13-41b0-bd27-ea8c368f9e28`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:14:30,777::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:14:30,857::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0378386 s, 27.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:14:31,475::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:14:32,869::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:14:33,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.315204 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:14:34,493::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:14:35,242::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:14:35,316::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310806 s, 32.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:14:36,230::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:14:36,238::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000628775 s, 6.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:14:37,329::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:14:37,411::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379516 s, 27.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:14:37,514::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:14:39,424::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:14:39,503::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356275 s, 28.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:14:39,756::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [28fd42b9] >Thread-66::DEBUG::2014-05-13 12:14:39,757::task::595::TaskManager.Task::(_updateState) Task=`c9b38c59-783c-49ad-9818-1fbd49e95ddc`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:14:39,757::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:14:39,757::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:14:39,757::task::1185::TaskManager.Task::(prepare) Task=`c9b38c59-783c-49ad-9818-1fbd49e95ddc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:14:39,758::task::595::TaskManager.Task::(_updateState) Task=`c9b38c59-783c-49ad-9818-1fbd49e95ddc`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:14:39,758::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:14:39,758::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:14:39,758::task::990::TaskManager.Task::(_decref) Task=`c9b38c59-783c-49ad-9818-1fbd49e95ddc`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:14:39,763::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [28fd42b9] >Thread-338::DEBUG::2014-05-13 12:14:39,763::task::595::TaskManager.Task::(_updateState) Task=`298441ea-46c7-420d-a25b-9922708aff6d`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:14:39,763::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:14:39,764::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`eedbd38b-cfcb-448f-9c99-4e5ed39574cf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:14:39,764::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:14:39,764::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:14:39,764::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`eedbd38b-cfcb-448f-9c99-4e5ed39574cf`::Granted request >Thread-338::DEBUG::2014-05-13 12:14:39,766::task::827::TaskManager.Task::(resourceAcquired) Task=`298441ea-46c7-420d-a25b-9922708aff6d`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:14:39,766::task::990::TaskManager.Task::(_decref) Task=`298441ea-46c7-420d-a25b-9922708aff6d`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:14:39,767::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:14:39,767::task::1185::TaskManager.Task::(prepare) Task=`298441ea-46c7-420d-a25b-9922708aff6d`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:14:39,767::task::595::TaskManager.Task::(_updateState) Task=`298441ea-46c7-420d-a25b-9922708aff6d`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:14:39,767::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:14:39,767::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:14:39,768::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:14:39,768::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:14:39,768::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:14:39,768::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:14:39,768::task::990::TaskManager.Task::(_decref) Task=`298441ea-46c7-420d-a25b-9922708aff6d`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:14:39,777::task::595::TaskManager.Task::(_updateState) Task=`24b15174-254b-4126-b500-0db932abe568`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:14:39,777::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:14:39,777::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000628775', 'lastCheck': '3.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:14:39,778::task::1185::TaskManager.Task::(prepare) Task=`24b15174-254b-4126-b500-0db932abe568`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000628775', 'lastCheck': '3.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:14:39,778::task::595::TaskManager.Task::(_updateState) Task=`24b15174-254b-4126-b500-0db932abe568`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:14:39,778::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:14:39,778::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:14:39,778::task::990::TaskManager.Task::(_decref) Task=`24b15174-254b-4126-b500-0db932abe568`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:14:40,538::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:14:41,516::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:14:41,590::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309005 s, 33.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:14:43,556::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:14:43,603::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:14:43,678::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321807 s, 31.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:14:45,691::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:14:46,094::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.35928 s, 2.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:14:46,245::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:14:46,253::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000688211 s, 6.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:14:46,576::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:14:48,107::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:14:48,189::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0398742 s, 25.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:14:49,594::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:14:49,804::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:14:49,805::task::595::TaskManager.Task::(_updateState) Task=`6a7efdc9-3ff4-4c47-8512-73ec69a26272`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:14:49,805::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:14:49,805::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:14:49,806::task::1185::TaskManager.Task::(prepare) Task=`6a7efdc9-3ff4-4c47-8512-73ec69a26272`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:14:49,806::task::595::TaskManager.Task::(_updateState) Task=`6a7efdc9-3ff4-4c47-8512-73ec69a26272`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:14:49,806::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:14:49,806::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:14:49,806::task::990::TaskManager.Task::(_decref) Task=`6a7efdc9-3ff4-4c47-8512-73ec69a26272`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:14:49,811::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:14:49,811::task::595::TaskManager.Task::(_updateState) Task=`ca0f1985-a8cc-4ee6-a209-b5ca3116b32e`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:14:49,811::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:14:49,812::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`07b4ad27-0b8d-4249-9c4e-1f6b5964dfdc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:14:49,812::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:14:49,812::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:14:49,812::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`07b4ad27-0b8d-4249-9c4e-1f6b5964dfdc`::Granted request >Thread-338::DEBUG::2014-05-13 12:14:49,812::task::827::TaskManager.Task::(resourceAcquired) Task=`ca0f1985-a8cc-4ee6-a209-b5ca3116b32e`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:14:49,813::task::990::TaskManager.Task::(_decref) Task=`ca0f1985-a8cc-4ee6-a209-b5ca3116b32e`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:14:49,813::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:14:49,813::task::1185::TaskManager.Task::(prepare) Task=`ca0f1985-a8cc-4ee6-a209-b5ca3116b32e`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:14:49,814::task::595::TaskManager.Task::(_updateState) Task=`ca0f1985-a8cc-4ee6-a209-b5ca3116b32e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:14:49,814::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:14:49,814::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:14:49,814::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:14:49,814::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:14:49,814::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:14:49,814::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:14:49,815::task::990::TaskManager.Task::(_decref) Task=`ca0f1985-a8cc-4ee6-a209-b5ca3116b32e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:14:50,202::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:14:50,276::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031259 s, 32.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:14:52,289::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:14:52,369::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357441 s, 28.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:14:52,613::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:14:54,382::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:14:54,462::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379596 s, 27.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:14:54,909::task::595::TaskManager.Task::(_updateState) Task=`a2d67b6f-3687-4ded-addd-28462578a90c`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:14:54,910::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:14:54,910::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000688211', 'lastCheck': '8.7', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:14:54,910::task::1185::TaskManager.Task::(prepare) Task=`a2d67b6f-3687-4ded-addd-28462578a90c`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000688211', 'lastCheck': '8.7', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:14:54,910::task::595::TaskManager.Task::(_updateState) Task=`a2d67b6f-3687-4ded-addd-28462578a90c`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:14:54,910::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:14:54,911::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:14:54,911::task::990::TaskManager.Task::(_decref) Task=`a2d67b6f-3687-4ded-addd-28462578a90c`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:14:55,632::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:14:56,260::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:14:56,268::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000732117 s, 5.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:14:56,475::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:14:56,850::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.333381 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:14:58,649::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:14:58,863::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:14:58,937::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315467 s, 32.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:14:59,837::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4a63f6c8] >Thread-66::DEBUG::2014-05-13 12:14:59,837::task::595::TaskManager.Task::(_updateState) Task=`a678bd02-54f7-4669-a9d8-b9c6a044f766`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:14:59,837::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:14:59,838::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:14:59,838::task::1185::TaskManager.Task::(prepare) Task=`a678bd02-54f7-4669-a9d8-b9c6a044f766`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:14:59,838::task::595::TaskManager.Task::(_updateState) Task=`a678bd02-54f7-4669-a9d8-b9c6a044f766`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:14:59,838::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:14:59,838::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:14:59,839::task::990::TaskManager.Task::(_decref) Task=`a678bd02-54f7-4669-a9d8-b9c6a044f766`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:14:59,843::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4a63f6c8] >Thread-338::DEBUG::2014-05-13 12:14:59,844::task::595::TaskManager.Task::(_updateState) Task=`594069fb-5666-4ae0-914f-146a930f7ef4`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:14:59,844::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:14:59,844::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9840ade4-5ac5-457a-a9e6-75adfc43df0b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:14:59,844::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:14:59,845::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:14:59,845::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9840ade4-5ac5-457a-a9e6-75adfc43df0b`::Granted request >Thread-338::DEBUG::2014-05-13 12:14:59,845::task::827::TaskManager.Task::(resourceAcquired) Task=`594069fb-5666-4ae0-914f-146a930f7ef4`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:14:59,845::task::990::TaskManager.Task::(_decref) Task=`594069fb-5666-4ae0-914f-146a930f7ef4`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:14:59,846::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:14:59,846::task::1185::TaskManager.Task::(prepare) Task=`594069fb-5666-4ae0-914f-146a930f7ef4`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:14:59,846::task::595::TaskManager.Task::(_updateState) Task=`594069fb-5666-4ae0-914f-146a930f7ef4`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:14:59,847::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:14:59,847::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:14:59,847::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:14:59,847::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:14:59,847::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:14:59,848::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:14:59,848::task::990::TaskManager.Task::(_decref) Task=`594069fb-5666-4ae0-914f-146a930f7ef4`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:15:00,950::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:15:01,236::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.208382 s, 4.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:15:01,668::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:15:03,248::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:15:03,321::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311252 s, 32.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:15:04,685::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:15:05,334::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:15:05,409::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309627 s, 33.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:15:06,272::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 41830dad-f012-4b4d-b636-863f66d5ca4b >Thread-322::DEBUG::2014-05-13 12:15:06,273::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-322::DEBUG::2014-05-13 12:15:06,276::blockSD::336::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:15:06,283::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000496035 s, 4.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:15:06,283::misc::287::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000496035 s, 4.1 MB/s'], size: 2048 >Thread-322::DEBUG::2014-05-13 12:15:06,284::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-322::WARNING::2014-05-13 12:15:06,284::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-322::DEBUG::2014-05-13 12:15:06,284::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-322::DEBUG::2014-05-13 12:15:06,284::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:15:06,285::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:15:06,285::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:15:06,285::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:15:06,285::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:15:06,286::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 12:15:06,558::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:15:06,560::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:15:06,560::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'LOCKPOLICY=', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', 'DESCRIPTION=ISCSI', 'LOGBLKSIZE=512', 'TYPE=ISCSI', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'PHYBLKSIZE=512', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', 'CLASS=Data', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'POOL_DESCRIPTION=DC_2', 'POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'MASTER_VERSION=1', 'ROLE=Master', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', '_SHA_CKSUM=28bb2d7cc60e0d779eb69fe93d050c16c769f069', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0'] >Thread-322::INFO::2014-05-13 12:15:06,561::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_imageNS already registered >Thread-322::INFO::2014-05-13 12:15:06,561::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_volumeNS already registered >Thread-322::INFO::2014-05-13 12:15:06,561::blockSD::452::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_lvmActivationNS already registered >Thread-322::DEBUG::2014-05-13 12:15:06,562::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 12:15:06,731::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:15:06,734::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:15:06,742::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000567326 s, 7.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:15:07,422::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:15:07,502::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361154 s, 28.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:15:07,704::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:15:09,515::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:15:09,608::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0493347 s, 20.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:15:09,870::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:15:09,871::task::595::TaskManager.Task::(_updateState) Task=`7e201187-cb9d-4677-81fc-7dac27e67213`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:15:09,871::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:15:09,871::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:15:09,872::task::1185::TaskManager.Task::(prepare) Task=`7e201187-cb9d-4677-81fc-7dac27e67213`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:15:09,872::task::595::TaskManager.Task::(_updateState) Task=`7e201187-cb9d-4677-81fc-7dac27e67213`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:15:09,872::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:15:09,872::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:15:09,873::task::990::TaskManager.Task::(_decref) Task=`7e201187-cb9d-4677-81fc-7dac27e67213`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:15:09,878::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:15:09,879::task::595::TaskManager.Task::(_updateState) Task=`55521ffe-8c0c-420a-aa6b-adb106f014bd`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:15:09,879::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:15:09,880::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3c067616-0bcb-4148-b42b-72b0367c91ae`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:15:09,880::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:15:09,880::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:15:09,880::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3c067616-0bcb-4148-b42b-72b0367c91ae`::Granted request >Thread-338::DEBUG::2014-05-13 12:15:09,881::task::827::TaskManager.Task::(resourceAcquired) Task=`55521ffe-8c0c-420a-aa6b-adb106f014bd`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:15:09,881::task::990::TaskManager.Task::(_decref) Task=`55521ffe-8c0c-420a-aa6b-adb106f014bd`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:15:09,882::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:15:09,882::task::1185::TaskManager.Task::(prepare) Task=`55521ffe-8c0c-420a-aa6b-adb106f014bd`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:15:09,883::task::595::TaskManager.Task::(_updateState) Task=`55521ffe-8c0c-420a-aa6b-adb106f014bd`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:15:09,883::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:15:09,883::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:15:09,883::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:15:09,884::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:15:09,884::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:15:09,884::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:15:09,884::task::990::TaskManager.Task::(_decref) Task=`55521ffe-8c0c-420a-aa6b-adb106f014bd`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:15:10,062::task::595::TaskManager.Task::(_updateState) Task=`af181c56-5aad-41bc-9d85-cdd8ac7f0737`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:15:10,063::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:15:10,063::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000567326', 'lastCheck': '3.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:15:10,063::task::1185::TaskManager.Task::(prepare) Task=`af181c56-5aad-41bc-9d85-cdd8ac7f0737`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000567326', 'lastCheck': '3.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:15:10,063::task::595::TaskManager.Task::(_updateState) Task=`af181c56-5aad-41bc-9d85-cdd8ac7f0737`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:15:10,063::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:15:10,064::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:15:10,064::task::990::TaskManager.Task::(_decref) Task=`af181c56-5aad-41bc-9d85-cdd8ac7f0737`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:15:10,723::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:15:11,621::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:15:11,695::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311382 s, 32.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:15:13,708::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:15:13,742::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:15:13,787::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0349288 s, 29.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:15:15,800::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:15:15,879::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358507 s, 28.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:15:16,749::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:15:16,757::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000602646 s, 6.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:15:16,761::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:15:17,892::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:15:17,971::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359044 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:15:19,780::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:15:19,903::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:15:19,904::task::595::TaskManager.Task::(_updateState) Task=`019f064b-e702-4b83-875d-fbaef441cbd4`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:15:19,904::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:15:19,904::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:15:19,904::task::1185::TaskManager.Task::(prepare) Task=`019f064b-e702-4b83-875d-fbaef441cbd4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:15:19,905::task::595::TaskManager.Task::(_updateState) Task=`019f064b-e702-4b83-875d-fbaef441cbd4`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:15:19,905::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:15:19,905::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:15:19,905::task::990::TaskManager.Task::(_decref) Task=`019f064b-e702-4b83-875d-fbaef441cbd4`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:15:19,912::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:15:19,912::task::595::TaskManager.Task::(_updateState) Task=`176a594e-0d33-4bd7-b678-fcdd1db0d03e`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:15:19,912::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:15:19,912::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3a8c126d-b3e2-4001-bd5e-23d486aba542`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:15:19,913::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:15:19,913::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:15:19,913::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3a8c126d-b3e2-4001-bd5e-23d486aba542`::Granted request >Thread-338::DEBUG::2014-05-13 12:15:19,913::task::827::TaskManager.Task::(resourceAcquired) Task=`176a594e-0d33-4bd7-b678-fcdd1db0d03e`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:15:19,913::task::990::TaskManager.Task::(_decref) Task=`176a594e-0d33-4bd7-b678-fcdd1db0d03e`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:15:19,914::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:15:19,914::task::1185::TaskManager.Task::(prepare) Task=`176a594e-0d33-4bd7-b678-fcdd1db0d03e`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:15:19,914::task::595::TaskManager.Task::(_updateState) Task=`176a594e-0d33-4bd7-b678-fcdd1db0d03e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:15:19,915::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:15:19,915::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:15:19,915::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:15:19,915::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:15:19,915::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:15:19,915::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:15:19,916::task::990::TaskManager.Task::(_decref) Task=`176a594e-0d33-4bd7-b678-fcdd1db0d03e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:15:19,984::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:15:20,059::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316163 s, 32.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:15:22,072::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:15:22,474::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.359976 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:15:22,807::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:15:24,487::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:15:24,567::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362299 s, 28.3 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:15:25,217::task::595::TaskManager.Task::(_updateState) Task=`c1194a33-bc9a-4dbf-9b17-5a044674c02b`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:15:25,218::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:15:25,218::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000602646', 'lastCheck': '8.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:15:25,218::task::1185::TaskManager.Task::(prepare) Task=`c1194a33-bc9a-4dbf-9b17-5a044674c02b`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000602646', 'lastCheck': '8.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:15:25,218::task::595::TaskManager.Task::(_updateState) Task=`c1194a33-bc9a-4dbf-9b17-5a044674c02b`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:15:25,218::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:15:25,218::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:15:25,219::task::990::TaskManager.Task::(_decref) Task=`c1194a33-bc9a-4dbf-9b17-5a044674c02b`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:15:25,826::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:15:26,580::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:15:26,654::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309225 s, 33.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:15:26,764::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:15:26,772::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000625571 s, 6.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:15:28,667::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:15:28,741::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310119 s, 33.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:15:28,846::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:15:29,935::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:15:29,936::task::595::TaskManager.Task::(_updateState) Task=`adc9d454-6ad5-47bb-b676-5efb01d44ebe`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:15:29,936::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:15:29,936::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:15:29,936::task::1185::TaskManager.Task::(prepare) Task=`adc9d454-6ad5-47bb-b676-5efb01d44ebe`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:15:29,937::task::595::TaskManager.Task::(_updateState) Task=`adc9d454-6ad5-47bb-b676-5efb01d44ebe`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:15:29,937::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:15:29,937::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:15:29,937::task::990::TaskManager.Task::(_decref) Task=`adc9d454-6ad5-47bb-b676-5efb01d44ebe`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:15:29,942::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:15:29,942::task::595::TaskManager.Task::(_updateState) Task=`bbfd01de-86db-4fc1-bbe0-0cd6c4fcde35`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:15:29,942::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:15:29,943::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b0cac766-c7a3-4f96-a62a-c9ee607b0040`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:15:29,943::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:15:29,943::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:15:29,944::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b0cac766-c7a3-4f96-a62a-c9ee607b0040`::Granted request >Thread-338::DEBUG::2014-05-13 12:15:29,944::task::827::TaskManager.Task::(resourceAcquired) Task=`bbfd01de-86db-4fc1-bbe0-0cd6c4fcde35`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:15:29,944::task::990::TaskManager.Task::(_decref) Task=`bbfd01de-86db-4fc1-bbe0-0cd6c4fcde35`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:15:29,945::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:15:29,945::task::1185::TaskManager.Task::(prepare) Task=`bbfd01de-86db-4fc1-bbe0-0cd6c4fcde35`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:15:29,945::task::595::TaskManager.Task::(_updateState) Task=`bbfd01de-86db-4fc1-bbe0-0cd6c4fcde35`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:15:29,945::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:15:29,946::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:15:29,946::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:15:29,946::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:15:29,946::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:15:29,946::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:15:29,947::task::990::TaskManager.Task::(_decref) Task=`bbfd01de-86db-4fc1-bbe0-0cd6c4fcde35`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:15:30,754::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:15:30,835::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379627 s, 27.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:15:31,864::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:15:32,848::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:15:32,932::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0422263 s, 24.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:15:34,882::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:15:34,945::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:15:35,021::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315855 s, 32.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:15:36,779::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:15:36,788::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000843044 s, 4.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:15:37,034::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:15:37,114::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0378545 s, 27.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:15:37,901::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:15:39,127::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:15:39,202::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314865 s, 32.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:15:39,971::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6849800f] >Thread-66::DEBUG::2014-05-13 12:15:39,972::task::595::TaskManager.Task::(_updateState) Task=`3e84c000-283b-4a89-8e00-5c27ed8c415b`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:15:39,972::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:15:39,973::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:15:39,973::task::1185::TaskManager.Task::(prepare) Task=`3e84c000-283b-4a89-8e00-5c27ed8c415b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:15:39,973::task::595::TaskManager.Task::(_updateState) Task=`3e84c000-283b-4a89-8e00-5c27ed8c415b`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:15:39,974::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:15:39,974::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:15:39,974::task::990::TaskManager.Task::(_decref) Task=`3e84c000-283b-4a89-8e00-5c27ed8c415b`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:15:39,981::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6849800f] >Thread-338::DEBUG::2014-05-13 12:15:39,981::task::595::TaskManager.Task::(_updateState) Task=`8067e093-7843-4699-b9c2-dc64d2a2edeb`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:15:39,981::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:15:39,982::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f8e53765-fe45-428a-9cb5-0d55041c02f3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:15:39,982::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:15:39,982::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:15:39,982::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f8e53765-fe45-428a-9cb5-0d55041c02f3`::Granted request >Thread-338::DEBUG::2014-05-13 12:15:39,983::task::827::TaskManager.Task::(resourceAcquired) Task=`8067e093-7843-4699-b9c2-dc64d2a2edeb`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:15:39,983::task::990::TaskManager.Task::(_decref) Task=`8067e093-7843-4699-b9c2-dc64d2a2edeb`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:15:39,984::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:15:39,984::task::1185::TaskManager.Task::(prepare) Task=`8067e093-7843-4699-b9c2-dc64d2a2edeb`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:15:39,984::task::595::TaskManager.Task::(_updateState) Task=`8067e093-7843-4699-b9c2-dc64d2a2edeb`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:15:39,984::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:15:39,984::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:15:39,985::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:15:39,985::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:15:39,985::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:15:39,985::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:15:39,985::task::990::TaskManager.Task::(_decref) Task=`8067e093-7843-4699-b9c2-dc64d2a2edeb`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:15:40,354::task::595::TaskManager.Task::(_updateState) Task=`8915ce75-89c5-45ef-9c34-4c3e2127c823`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:15:40,354::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:15:40,355::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000843044', 'lastCheck': '3.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:15:40,355::task::1185::TaskManager.Task::(prepare) Task=`8915ce75-89c5-45ef-9c34-4c3e2127c823`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000843044', 'lastCheck': '3.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:15:40,355::task::595::TaskManager.Task::(_updateState) Task=`8915ce75-89c5-45ef-9c34-4c3e2127c823`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:15:40,355::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:15:40,355::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:15:40,355::task::990::TaskManager.Task::(_decref) Task=`8915ce75-89c5-45ef-9c34-4c3e2127c823`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:15:40,919::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:15:41,215::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:15:41,290::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0308164 s, 33.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:15:43,373::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:15:43,458::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365262 s, 28.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:15:43,943::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:15:45,471::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:15:45,551::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036948 s, 27.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:15:46,794::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:15:46,802::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000627809 s, 6.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:15:46,962::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:15:47,564::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:15:47,974::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.366494 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:15:49,980::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:15:49,987::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 12:15:50,004::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:15:50,006::task::595::TaskManager.Task::(_updateState) Task=`2a4da210-a1cd-4480-ba77-1006e7a4cfca`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:15:50,007::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:15:50,007::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:15:50,008::task::1185::TaskManager.Task::(prepare) Task=`2a4da210-a1cd-4480-ba77-1006e7a4cfca`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:15:50,008::task::595::TaskManager.Task::(_updateState) Task=`2a4da210-a1cd-4480-ba77-1006e7a4cfca`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:15:50,008::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:15:50,008::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:15:50,009::task::990::TaskManager.Task::(_decref) Task=`2a4da210-a1cd-4480-ba77-1006e7a4cfca`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:15:50,014::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:15:50,015::task::595::TaskManager.Task::(_updateState) Task=`ce681894-e253-4527-a36f-ee2977209428`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:15:50,015::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:15:50,015::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`05c42e2d-be36-475b-bbf9-1af5f577e8c9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:15:50,016::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:15:50,016::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:15:50,016::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`05c42e2d-be36-475b-bbf9-1af5f577e8c9`::Granted request >Thread-338::DEBUG::2014-05-13 12:15:50,017::task::827::TaskManager.Task::(resourceAcquired) Task=`ce681894-e253-4527-a36f-ee2977209428`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:15:50,017::task::990::TaskManager.Task::(_decref) Task=`ce681894-e253-4527-a36f-ee2977209428`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:15:50,018::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:15:50,018::task::1185::TaskManager.Task::(prepare) Task=`ce681894-e253-4527-a36f-ee2977209428`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:15:50,019::task::595::TaskManager.Task::(_updateState) Task=`ce681894-e253-4527-a36f-ee2977209428`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:15:50,019::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:15:50,019::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:15:50,019::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:15:50,020::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:15:50,020::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:15:50,020::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:15:50,020::task::990::TaskManager.Task::(_decref) Task=`ce681894-e253-4527-a36f-ee2977209428`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:15:50,074::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0440214 s, 23.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:15:52,087::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:15:52,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0367235 s, 27.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:15:52,997::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:15:54,180::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:15:54,257::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323015 s, 31.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:15:55,489::task::595::TaskManager.Task::(_updateState) Task=`ef2e009f-08e6-4a8f-a7c4-a70625223c0b`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:15:55,489::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:15:55,490::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000627809', 'lastCheck': '8.7', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:15:55,490::task::1185::TaskManager.Task::(prepare) Task=`ef2e009f-08e6-4a8f-a7c4-a70625223c0b`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000627809', 'lastCheck': '8.7', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:15:55,490::task::595::TaskManager.Task::(_updateState) Task=`ef2e009f-08e6-4a8f-a7c4-a70625223c0b`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:15:55,490::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:15:55,490::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:15:55,490::task::990::TaskManager.Task::(_decref) Task=`ef2e009f-08e6-4a8f-a7c4-a70625223c0b`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:15:56,023::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:15:56,270::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:15:56,353::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0372111 s, 27.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:15:56,809::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:15:56,817::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000648776 s, 6.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:15:58,366::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:15:58,445::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360279 s, 28.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:15:59,042::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:16:00,041::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17998964] >Thread-66::DEBUG::2014-05-13 12:16:00,042::task::595::TaskManager.Task::(_updateState) Task=`04238fdc-3e7c-45d1-bcbd-eb18204bab89`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:16:00,042::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:16:00,043::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:16:00,043::task::1185::TaskManager.Task::(prepare) Task=`04238fdc-3e7c-45d1-bcbd-eb18204bab89`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:16:00,043::task::595::TaskManager.Task::(_updateState) Task=`04238fdc-3e7c-45d1-bcbd-eb18204bab89`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:16:00,043::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:16:00,043::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:16:00,044::task::990::TaskManager.Task::(_decref) Task=`04238fdc-3e7c-45d1-bcbd-eb18204bab89`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:16:00,049::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17998964] >Thread-338::DEBUG::2014-05-13 12:16:00,049::task::595::TaskManager.Task::(_updateState) Task=`e25bc4fa-d86a-4dca-9d69-4abbbfbde66f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:16:00,049::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:16:00,050::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`927eac8d-6d38-45d0-8d04-999b54195fb6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:16:00,050::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:16:00,050::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:16:00,050::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`927eac8d-6d38-45d0-8d04-999b54195fb6`::Granted request >Thread-338::DEBUG::2014-05-13 12:16:00,051::task::827::TaskManager.Task::(resourceAcquired) Task=`e25bc4fa-d86a-4dca-9d69-4abbbfbde66f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:16:00,051::task::990::TaskManager.Task::(_decref) Task=`e25bc4fa-d86a-4dca-9d69-4abbbfbde66f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:16:00,051::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:16:00,052::task::1185::TaskManager.Task::(prepare) Task=`e25bc4fa-d86a-4dca-9d69-4abbbfbde66f`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:16:00,052::task::595::TaskManager.Task::(_updateState) Task=`e25bc4fa-d86a-4dca-9d69-4abbbfbde66f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:16:00,052::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:16:00,052::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:16:00,052::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:16:00,053::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:16:00,053::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:16:00,053::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:16:00,053::task::990::TaskManager.Task::(_decref) Task=`e25bc4fa-d86a-4dca-9d69-4abbbfbde66f`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:16:00,458::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:00,550::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0497058 s, 20.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:16:02,059::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:16:02,563::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:02,639::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0345653 s, 29.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:16:04,652::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:04,727::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035947 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:16:05,076::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:16:06,735::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:06,816::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0366009 s, 28.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:16:06,840::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:16:06,857::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000688948 s, 5.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:16:08,095::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:16:08,879::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:08,954::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315559 s, 32.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:16:10,074::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [303c64a0] >Thread-66::DEBUG::2014-05-13 12:16:10,099::task::595::TaskManager.Task::(_updateState) Task=`97e19123-91b2-4e0c-beb7-7174aa347138`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:16:10,099::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:16:10,099::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:16:10,099::task::1185::TaskManager.Task::(prepare) Task=`97e19123-91b2-4e0c-beb7-7174aa347138`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:16:10,100::task::595::TaskManager.Task::(_updateState) Task=`97e19123-91b2-4e0c-beb7-7174aa347138`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:16:10,100::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:16:10,100::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:16:10,100::task::990::TaskManager.Task::(_decref) Task=`97e19123-91b2-4e0c-beb7-7174aa347138`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:16:10,105::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [303c64a0] >Thread-338::DEBUG::2014-05-13 12:16:10,105::task::595::TaskManager.Task::(_updateState) Task=`eab658cd-1349-4dc7-9e6f-df2873f6170a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:16:10,106::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:16:10,106::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1488a927-d2db-4cfe-b1d7-dda5630e075e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:16:10,106::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:16:10,106::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:16:10,107::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1488a927-d2db-4cfe-b1d7-dda5630e075e`::Granted request >Thread-338::DEBUG::2014-05-13 12:16:10,107::task::827::TaskManager.Task::(resourceAcquired) Task=`eab658cd-1349-4dc7-9e6f-df2873f6170a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:16:10,107::task::990::TaskManager.Task::(_decref) Task=`eab658cd-1349-4dc7-9e6f-df2873f6170a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:16:10,108::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:16:10,108::task::1185::TaskManager.Task::(prepare) Task=`eab658cd-1349-4dc7-9e6f-df2873f6170a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:16:10,108::task::595::TaskManager.Task::(_updateState) Task=`eab658cd-1349-4dc7-9e6f-df2873f6170a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:16:10,108::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:16:10,109::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:16:10,109::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:16:10,109::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:16:10,109::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:16:10,109::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:16:10,110::task::990::TaskManager.Task::(_decref) Task=`eab658cd-1349-4dc7-9e6f-df2873f6170a`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:16:10,629::task::595::TaskManager.Task::(_updateState) Task=`30887575-7779-4713-a188-e07ebe7d4b27`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:16:10,629::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:16:10,630::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000688948', 'lastCheck': '3.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:16:10,630::task::1185::TaskManager.Task::(prepare) Task=`30887575-7779-4713-a188-e07ebe7d4b27`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000688948', 'lastCheck': '3.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:16:10,630::task::595::TaskManager.Task::(_updateState) Task=`30887575-7779-4713-a188-e07ebe7d4b27`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:16:10,630::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:16:10,630::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:16:10,631::task::990::TaskManager.Task::(_decref) Task=`30887575-7779-4713-a188-e07ebe7d4b27`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:16:10,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:11,042::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318746 s, 32.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:16:11,114::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:16:13,055::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:13,136::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0371613 s, 27.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:16:14,140::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:16:15,149::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:15,241::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0480992 s, 21.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:16:16,877::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:16:16,885::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000735305 s, 5.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:16:17,158::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:16:17,254::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:17,335::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0368581 s, 27.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:16:19,348::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:19,429::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037231 s, 27.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:16:20,132::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:16:20,132::task::595::TaskManager.Task::(_updateState) Task=`7d7e0670-27a1-4258-ae27-b6a845ef6088`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:16:20,133::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:16:20,133::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:16:20,133::task::1185::TaskManager.Task::(prepare) Task=`7d7e0670-27a1-4258-ae27-b6a845ef6088`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:16:20,133::task::595::TaskManager.Task::(_updateState) Task=`7d7e0670-27a1-4258-ae27-b6a845ef6088`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:16:20,133::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:16:20,134::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:16:20,134::task::990::TaskManager.Task::(_decref) Task=`7d7e0670-27a1-4258-ae27-b6a845ef6088`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:16:20,138::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:16:20,138::task::595::TaskManager.Task::(_updateState) Task=`eb3ad1b5-2aaf-47a9-a1f1-0b8532830388`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:16:20,139::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:16:20,139::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3c2f621e-8938-429b-8191-0d3038ce8644`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:16:20,139::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:16:20,139::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:16:20,140::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3c2f621e-8938-429b-8191-0d3038ce8644`::Granted request >Thread-338::DEBUG::2014-05-13 12:16:20,140::task::827::TaskManager.Task::(resourceAcquired) Task=`eb3ad1b5-2aaf-47a9-a1f1-0b8532830388`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:16:20,140::task::990::TaskManager.Task::(_decref) Task=`eb3ad1b5-2aaf-47a9-a1f1-0b8532830388`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:16:20,141::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:16:20,141::task::1185::TaskManager.Task::(prepare) Task=`eb3ad1b5-2aaf-47a9-a1f1-0b8532830388`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:16:20,141::task::595::TaskManager.Task::(_updateState) Task=`eb3ad1b5-2aaf-47a9-a1f1-0b8532830388`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:16:20,141::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:16:20,141::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:16:20,142::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:16:20,142::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:16:20,142::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:16:20,142::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:16:20,142::task::990::TaskManager.Task::(_decref) Task=`eb3ad1b5-2aaf-47a9-a1f1-0b8532830388`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:16:20,175::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:16:21,442::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:21,674::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321776 s, 31.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:16:23,192::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:16:23,687::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:23,764::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0335866 s, 30.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:16:25,758::task::595::TaskManager.Task::(_updateState) Task=`caacf43d-1945-4491-ae05-8d0e6ec9c4bd`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:16:25,758::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:16:25,758::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000735305', 'lastCheck': '8.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:16:25,758::task::1185::TaskManager.Task::(prepare) Task=`caacf43d-1945-4491-ae05-8d0e6ec9c4bd`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000735305', 'lastCheck': '8.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:16:25,759::task::595::TaskManager.Task::(_updateState) Task=`caacf43d-1945-4491-ae05-8d0e6ec9c4bd`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:16:25,759::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:16:25,759::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:16:25,759::task::990::TaskManager.Task::(_decref) Task=`caacf43d-1945-4491-ae05-8d0e6ec9c4bd`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:16:25,777::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:25,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.030227 s, 33.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:16:26,215::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:16:26,892::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:16:26,899::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000630588 s, 6.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:16:27,864::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:27,947::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0392234 s, 26.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:16:29,233::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:16:29,960::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:30,051::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0490118 s, 20.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:16:30,159::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:16:30,160::task::595::TaskManager.Task::(_updateState) Task=`5c2d022a-3361-4741-ae55-6162ee04df5f`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:16:30,160::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:16:30,160::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:16:30,160::task::1185::TaskManager.Task::(prepare) Task=`5c2d022a-3361-4741-ae55-6162ee04df5f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:16:30,161::task::595::TaskManager.Task::(_updateState) Task=`5c2d022a-3361-4741-ae55-6162ee04df5f`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:16:30,161::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:16:30,161::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:16:30,161::task::990::TaskManager.Task::(_decref) Task=`5c2d022a-3361-4741-ae55-6162ee04df5f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:16:30,165::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:16:30,166::task::595::TaskManager.Task::(_updateState) Task=`7f2cc360-dc95-438f-a224-0b0d2eac28d9`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:16:30,166::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:16:30,166::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`dcdcd10c-6367-4945-9829-888927c564a8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:16:30,167::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:16:30,167::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:16:30,167::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`dcdcd10c-6367-4945-9829-888927c564a8`::Granted request >Thread-338::DEBUG::2014-05-13 12:16:30,167::task::827::TaskManager.Task::(resourceAcquired) Task=`7f2cc360-dc95-438f-a224-0b0d2eac28d9`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:16:30,168::task::990::TaskManager.Task::(_decref) Task=`7f2cc360-dc95-438f-a224-0b0d2eac28d9`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:16:30,168::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:16:30,169::task::1185::TaskManager.Task::(prepare) Task=`7f2cc360-dc95-438f-a224-0b0d2eac28d9`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:16:30,169::task::595::TaskManager.Task::(_updateState) Task=`7f2cc360-dc95-438f-a224-0b0d2eac28d9`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:16:30,169::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:16:30,169::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:16:30,169::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:16:30,169::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:16:30,170::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:16:30,170::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:16:30,170::task::990::TaskManager.Task::(_decref) Task=`7f2cc360-dc95-438f-a224-0b0d2eac28d9`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:16:32,064::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:32,138::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309753 s, 33.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:16:32,250::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:16:34,151::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:34,233::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0380789 s, 26.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:16:35,268::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:16:36,246::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:36,321::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322517 s, 31.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:16:36,910::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:16:36,918::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000713184 s, 5.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:16:38,288::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:16:38,334::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:38,407::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.030356 s, 33.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:16:40,191::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [50b954f9] >Thread-66::DEBUG::2014-05-13 12:16:40,192::task::595::TaskManager.Task::(_updateState) Task=`6114f3df-17a7-4587-b398-73162b282949`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:16:40,192::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:16:40,193::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:16:40,193::task::1185::TaskManager.Task::(prepare) Task=`6114f3df-17a7-4587-b398-73162b282949`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:16:40,193::task::595::TaskManager.Task::(_updateState) Task=`6114f3df-17a7-4587-b398-73162b282949`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:16:40,193::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:16:40,193::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:16:40,193::task::990::TaskManager.Task::(_decref) Task=`6114f3df-17a7-4587-b398-73162b282949`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:16:40,198::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [50b954f9] >Thread-338::DEBUG::2014-05-13 12:16:40,198::task::595::TaskManager.Task::(_updateState) Task=`37c0c3de-2c19-4a15-959e-7b199fd47351`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:16:40,198::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:16:40,199::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`100a8e0e-e3f7-4086-80cb-8605e4b49cb0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:16:40,199::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:16:40,199::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:16:40,200::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`100a8e0e-e3f7-4086-80cb-8605e4b49cb0`::Granted request >Thread-338::DEBUG::2014-05-13 12:16:40,200::task::827::TaskManager.Task::(resourceAcquired) Task=`37c0c3de-2c19-4a15-959e-7b199fd47351`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:16:40,200::task::990::TaskManager.Task::(_decref) Task=`37c0c3de-2c19-4a15-959e-7b199fd47351`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:16:40,201::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:16:40,201::task::1185::TaskManager.Task::(prepare) Task=`37c0c3de-2c19-4a15-959e-7b199fd47351`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:16:40,201::task::595::TaskManager.Task::(_updateState) Task=`37c0c3de-2c19-4a15-959e-7b199fd47351`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:16:40,201::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:16:40,202::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:16:40,202::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:16:40,202::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:16:40,202::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:16:40,202::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:16:40,203::task::990::TaskManager.Task::(_decref) Task=`37c0c3de-2c19-4a15-959e-7b199fd47351`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:16:40,432::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:40,514::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.039191 s, 26.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:16:40,892::task::595::TaskManager.Task::(_updateState) Task=`2281bfb0-7bbc-4033-a533-5f17d776dc03`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:16:40,892::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:16:40,892::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000713184', 'lastCheck': '4.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:16:40,892::task::1185::TaskManager.Task::(prepare) Task=`2281bfb0-7bbc-4033-a533-5f17d776dc03`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000713184', 'lastCheck': '4.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:16:40,893::task::595::TaskManager.Task::(_updateState) Task=`2281bfb0-7bbc-4033-a533-5f17d776dc03`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:16:40,893::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:16:40,893::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:16:40,893::task::990::TaskManager.Task::(_decref) Task=`2281bfb0-7bbc-4033-a533-5f17d776dc03`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:16:41,307::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:16:42,527::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:42,936::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.366232 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:16:44,324::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:16:44,949::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:45,029::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352726 s, 29.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:16:46,925::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:16:46,932::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000561607 s, 7.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:16:47,042::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:47,120::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356571 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:16:47,348::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:16:49,133::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:49,213::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364059 s, 28.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:16:50,220::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5fad7764] >Thread-66::DEBUG::2014-05-13 12:16:50,221::task::595::TaskManager.Task::(_updateState) Task=`58683c53-3c1b-47f8-a1d8-3bfac6cb6d20`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:16:50,221::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:16:50,221::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:16:50,221::task::1185::TaskManager.Task::(prepare) Task=`58683c53-3c1b-47f8-a1d8-3bfac6cb6d20`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:16:50,222::task::595::TaskManager.Task::(_updateState) Task=`58683c53-3c1b-47f8-a1d8-3bfac6cb6d20`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:16:50,222::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:16:50,222::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:16:50,222::task::990::TaskManager.Task::(_decref) Task=`58683c53-3c1b-47f8-a1d8-3bfac6cb6d20`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:16:50,227::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5fad7764] >Thread-338::DEBUG::2014-05-13 12:16:50,227::task::595::TaskManager.Task::(_updateState) Task=`8250dd4e-fd38-48e5-abce-c721532832e4`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:16:50,227::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:16:50,228::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`895971c0-8c68-4002-b558-be1ff47c908c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:16:50,228::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:16:50,228::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:16:50,228::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`895971c0-8c68-4002-b558-be1ff47c908c`::Granted request >Thread-338::DEBUG::2014-05-13 12:16:50,229::task::827::TaskManager.Task::(resourceAcquired) Task=`8250dd4e-fd38-48e5-abce-c721532832e4`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:16:50,229::task::990::TaskManager.Task::(_decref) Task=`8250dd4e-fd38-48e5-abce-c721532832e4`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:16:50,229::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:16:50,229::task::1185::TaskManager.Task::(prepare) Task=`8250dd4e-fd38-48e5-abce-c721532832e4`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:16:50,230::task::595::TaskManager.Task::(_updateState) Task=`8250dd4e-fd38-48e5-abce-c721532832e4`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:16:50,230::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:16:50,230::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:16:50,230::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:16:50,230::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:16:50,230::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:16:50,231::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:16:50,231::task::990::TaskManager.Task::(_decref) Task=`8250dd4e-fd38-48e5-abce-c721532832e4`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:16:50,366::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:16:51,226::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:51,301::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310585 s, 33.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:16:53,314::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:16:53,384::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:16:53,393::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313867 s, 32.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:16:55,406::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:55,497::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0484615 s, 21.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:16:56,027::task::595::TaskManager.Task::(_updateState) Task=`297ae740-2438-4aee-bb47-b1ba9a0c15cb`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:16:56,027::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:16:56,028::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000561607', 'lastCheck': '9.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:16:56,028::task::1185::TaskManager.Task::(prepare) Task=`297ae740-2438-4aee-bb47-b1ba9a0c15cb`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000561607', 'lastCheck': '9.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:16:56,028::task::595::TaskManager.Task::(_updateState) Task=`297ae740-2438-4aee-bb47-b1ba9a0c15cb`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:16:56,028::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:16:56,028::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:16:56,029::task::990::TaskManager.Task::(_decref) Task=`297ae740-2438-4aee-bb47-b1ba9a0c15cb`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:16:56,402::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:16:56,939::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:16:56,947::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000590618 s, 6.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:16:57,511::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:57,592::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0385224 s, 26.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:16:59,426::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:16:59,604::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:16:59,686::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0351169 s, 29.2 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:17:00,250::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:17:00,251::task::595::TaskManager.Task::(_updateState) Task=`5159b21c-a30b-40b4-a0dd-47dd68032c9f`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:17:00,251::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:17:00,251::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:17:00,251::task::1185::TaskManager.Task::(prepare) Task=`5159b21c-a30b-40b4-a0dd-47dd68032c9f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:17:00,252::task::595::TaskManager.Task::(_updateState) Task=`5159b21c-a30b-40b4-a0dd-47dd68032c9f`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:17:00,252::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:17:00,252::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:17:00,252::task::990::TaskManager.Task::(_decref) Task=`5159b21c-a30b-40b4-a0dd-47dd68032c9f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:17:00,257::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:17:00,257::task::595::TaskManager.Task::(_updateState) Task=`6670c52e-e83f-4cdb-9887-3008746b3a74`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:17:00,257::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:17:00,259::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`7a97a070-3800-478a-b076-cadbaaf52f8d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:17:00,259::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:17:00,259::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:17:00,259::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`7a97a070-3800-478a-b076-cadbaaf52f8d`::Granted request >Thread-338::DEBUG::2014-05-13 12:17:00,260::task::827::TaskManager.Task::(resourceAcquired) Task=`6670c52e-e83f-4cdb-9887-3008746b3a74`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:17:00,260::task::990::TaskManager.Task::(_decref) Task=`6670c52e-e83f-4cdb-9887-3008746b3a74`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:17:00,261::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:17:00,261::task::1185::TaskManager.Task::(prepare) Task=`6670c52e-e83f-4cdb-9887-3008746b3a74`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:17:00,261::task::595::TaskManager.Task::(_updateState) Task=`6670c52e-e83f-4cdb-9887-3008746b3a74`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:17:00,261::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:17:00,261::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:17:00,261::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:17:00,262::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:17:00,262::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:17:00,262::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:17:00,262::task::990::TaskManager.Task::(_decref) Task=`6670c52e-e83f-4cdb-9887-3008746b3a74`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:17:01,699::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:17:01,805::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0385932 s, 26.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:17:02,446::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:17:03,813::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:17:03,887::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305515 s, 33.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:17:05,465::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:17:05,900::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:17:05,974::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313239 s, 32.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:17:06,954::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:17:06,962::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000671431 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:17:07,987::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:17:08,067::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0368598 s, 27.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:17:08,492::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:17:10,080::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:17:10,161::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0388757 s, 26.3 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:17:10,280::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:17:10,281::task::595::TaskManager.Task::(_updateState) Task=`caafc550-28ec-4569-841e-b9a7150c613d`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:17:10,281::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:17:10,281::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:17:10,281::task::1185::TaskManager.Task::(prepare) Task=`caafc550-28ec-4569-841e-b9a7150c613d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:17:10,282::task::595::TaskManager.Task::(_updateState) Task=`caafc550-28ec-4569-841e-b9a7150c613d`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:17:10,282::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:17:10,282::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:17:10,282::task::990::TaskManager.Task::(_decref) Task=`caafc550-28ec-4569-841e-b9a7150c613d`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:17:10,287::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:17:10,287::task::595::TaskManager.Task::(_updateState) Task=`d50af88f-27e7-4479-8599-3f51f9c5d4ba`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:17:10,287::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:17:10,288::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9536e718-e1fc-4610-bff0-315401c71c9b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:17:10,288::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:17:10,288::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:17:10,289::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9536e718-e1fc-4610-bff0-315401c71c9b`::Granted request >Thread-338::DEBUG::2014-05-13 12:17:10,289::task::827::TaskManager.Task::(resourceAcquired) Task=`d50af88f-27e7-4479-8599-3f51f9c5d4ba`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:17:10,289::task::990::TaskManager.Task::(_decref) Task=`d50af88f-27e7-4479-8599-3f51f9c5d4ba`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:17:10,290::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:17:10,290::task::1185::TaskManager.Task::(prepare) Task=`d50af88f-27e7-4479-8599-3f51f9c5d4ba`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:17:10,290::task::595::TaskManager.Task::(_updateState) Task=`d50af88f-27e7-4479-8599-3f51f9c5d4ba`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:17:10,290::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:17:10,291::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:17:10,291::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:17:10,291::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:17:10,291::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:17:10,291::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:17:10,292::task::990::TaskManager.Task::(_decref) Task=`d50af88f-27e7-4479-8599-3f51f9c5d4ba`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:17:11,154::task::595::TaskManager.Task::(_updateState) Task=`5bc95fa9-57cd-4e52-8a11-6eedadb9d4a6`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:17:11,154::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:17:11,154::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000671431', 'lastCheck': '4.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:17:11,154::task::1185::TaskManager.Task::(prepare) Task=`5bc95fa9-57cd-4e52-8a11-6eedadb9d4a6`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000671431', 'lastCheck': '4.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:17:11,155::task::595::TaskManager.Task::(_updateState) Task=`5bc95fa9-57cd-4e52-8a11-6eedadb9d4a6`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:17:11,155::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:17:11,155::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:17:11,155::task::990::TaskManager.Task::(_decref) Task=`5bc95fa9-57cd-4e52-8a11-6eedadb9d4a6`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:17:11,517::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:17:12,174::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:17:12,261::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0428509 s, 23.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:17:14,275::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:17:14,351::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.033719 s, 30.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:17:14,535::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:17:16,363::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:17:16,444::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0372781 s, 27.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:17:16,997::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:17:17,005::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000669948 s, 6.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:17:17,554::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:17:18,457::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:17:18,548::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359597 s, 28.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:17:20,308::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:17:20,309::task::595::TaskManager.Task::(_updateState) Task=`2e477bf6-c24f-490d-8d54-f1346767db74`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:17:20,309::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:17:20,310::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:17:20,310::task::1185::TaskManager.Task::(prepare) Task=`2e477bf6-c24f-490d-8d54-f1346767db74`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:17:20,310::task::595::TaskManager.Task::(_updateState) Task=`2e477bf6-c24f-490d-8d54-f1346767db74`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:17:20,310::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:17:20,310::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:17:20,310::task::990::TaskManager.Task::(_decref) Task=`2e477bf6-c24f-490d-8d54-f1346767db74`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:17:20,315::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:17:20,315::task::595::TaskManager.Task::(_updateState) Task=`1fb93662-224c-4fca-b434-24a082b13807`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:17:20,315::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:17:20,316::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a3256aac-a3eb-45cb-861f-123c5cb48f0b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:17:20,316::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:17:20,316::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:17:20,316::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a3256aac-a3eb-45cb-861f-123c5cb48f0b`::Granted request >Thread-338::DEBUG::2014-05-13 12:17:20,317::task::827::TaskManager.Task::(resourceAcquired) Task=`1fb93662-224c-4fca-b434-24a082b13807`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:17:20,317::task::990::TaskManager.Task::(_decref) Task=`1fb93662-224c-4fca-b434-24a082b13807`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:17:20,317::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:17:20,318::task::1185::TaskManager.Task::(prepare) Task=`1fb93662-224c-4fca-b434-24a082b13807`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:17:20,318::task::595::TaskManager.Task::(_updateState) Task=`1fb93662-224c-4fca-b434-24a082b13807`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:17:20,318::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:17:20,318::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:17:20,318::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:17:20,318::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:17:20,319::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:17:20,319::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:17:20,319::task::990::TaskManager.Task::(_decref) Task=`1fb93662-224c-4fca-b434-24a082b13807`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:17:20,562::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:17:20,579::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:17:20,639::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0348361 s, 29.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:17:22,652::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:17:22,732::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383377 s, 26.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:17:23,597::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:17:24,745::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:17:24,831::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0431278 s, 23.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:17:26,291::task::595::TaskManager.Task::(_updateState) Task=`0a06e59c-b50b-4b4b-8039-677f78212a37`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:17:26,292::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:17:26,292::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000669948', 'lastCheck': '9.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:17:26,292::task::1185::TaskManager.Task::(prepare) Task=`0a06e59c-b50b-4b4b-8039-677f78212a37`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000669948', 'lastCheck': '9.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:17:26,292::task::595::TaskManager.Task::(_updateState) Task=`0a06e59c-b50b-4b4b-8039-677f78212a37`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:17:26,292::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:17:26,292::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:17:26,293::task::990::TaskManager.Task::(_decref) Task=`0a06e59c-b50b-4b4b-8039-677f78212a37`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:17:26,617::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:17:26,844::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 12:17:27,012::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:17:27,163::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.138967 s, 29.5 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:17:27,241::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.355347 s, 2.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:17:29,254::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:17:29,352::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0390079 s, 26.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:17:29,638::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:17:30,337::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:17:30,338::task::595::TaskManager.Task::(_updateState) Task=`77d764bb-85b9-4f39-8f19-4f81543ba814`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:17:30,338::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:17:30,338::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:17:30,339::task::1185::TaskManager.Task::(prepare) Task=`77d764bb-85b9-4f39-8f19-4f81543ba814`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:17:30,339::task::595::TaskManager.Task::(_updateState) Task=`77d764bb-85b9-4f39-8f19-4f81543ba814`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:17:30,339::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:17:30,339::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:17:30,339::task::990::TaskManager.Task::(_decref) Task=`77d764bb-85b9-4f39-8f19-4f81543ba814`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:17:30,344::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:17:30,344::task::595::TaskManager.Task::(_updateState) Task=`9310a2da-31df-4622-8301-bf319cb7afc6`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:17:30,344::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:17:30,345::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1730cb08-6fab-4a4a-ba53-dfd0a71fc305`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:17:30,345::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:17:30,345::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:17:30,345::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1730cb08-6fab-4a4a-ba53-dfd0a71fc305`::Granted request >Thread-338::DEBUG::2014-05-13 12:17:30,346::task::827::TaskManager.Task::(resourceAcquired) Task=`9310a2da-31df-4622-8301-bf319cb7afc6`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:17:30,346::task::990::TaskManager.Task::(_decref) Task=`9310a2da-31df-4622-8301-bf319cb7afc6`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:17:30,346::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:17:30,347::task::1185::TaskManager.Task::(prepare) Task=`9310a2da-31df-4622-8301-bf319cb7afc6`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:17:30,347::task::595::TaskManager.Task::(_updateState) Task=`9310a2da-31df-4622-8301-bf319cb7afc6`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:17:30,347::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:17:30,347::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:17:30,347::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:17:30,348::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:17:30,348::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:17:30,348::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:17:30,348::task::990::TaskManager.Task::(_decref) Task=`9310a2da-31df-4622-8301-bf319cb7afc6`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:17:31,370::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:17:31,450::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382523 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:17:32,656::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:17:33,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:17:33,538::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316154 s, 32.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:17:35,551::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:17:35,624::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307336 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:17:35,673::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:17:37,183::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:17:37,191::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000640246 s, 6.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:17:37,637::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:17:37,731::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.051545 s, 19.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:17:38,692::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:17:39,744::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:17:39,827::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320504 s, 31.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:17:40,379::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:17:40,380::task::595::TaskManager.Task::(_updateState) Task=`80e35677-cb93-4ce5-bdaf-f14358c0ec2d`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:17:40,380::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:17:40,380::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:17:40,380::task::1185::TaskManager.Task::(prepare) Task=`80e35677-cb93-4ce5-bdaf-f14358c0ec2d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:17:40,381::task::595::TaskManager.Task::(_updateState) Task=`80e35677-cb93-4ce5-bdaf-f14358c0ec2d`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:17:40,381::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:17:40,381::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:17:40,381::task::990::TaskManager.Task::(_decref) Task=`80e35677-cb93-4ce5-bdaf-f14358c0ec2d`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:17:40,386::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:17:40,386::task::595::TaskManager.Task::(_updateState) Task=`a8fa0c13-c32b-4c69-9df7-ad13d7dd1e76`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:17:40,386::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:17:40,387::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`031011dd-e329-4600-99cc-46d9530f7cd2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:17:40,387::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:17:40,387::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:17:40,387::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`031011dd-e329-4600-99cc-46d9530f7cd2`::Granted request >Thread-338::DEBUG::2014-05-13 12:17:40,388::task::827::TaskManager.Task::(resourceAcquired) Task=`a8fa0c13-c32b-4c69-9df7-ad13d7dd1e76`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:17:40,388::task::990::TaskManager.Task::(_decref) Task=`a8fa0c13-c32b-4c69-9df7-ad13d7dd1e76`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:17:40,389::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:17:40,389::task::1185::TaskManager.Task::(prepare) Task=`a8fa0c13-c32b-4c69-9df7-ad13d7dd1e76`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:17:40,389::task::595::TaskManager.Task::(_updateState) Task=`a8fa0c13-c32b-4c69-9df7-ad13d7dd1e76`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:17:40,390::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:17:40,390::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:17:40,390::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:17:40,390::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:17:40,390::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:17:40,390::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:17:40,391::task::990::TaskManager.Task::(_decref) Task=`a8fa0c13-c32b-4c69-9df7-ad13d7dd1e76`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:17:41,431::task::595::TaskManager.Task::(_updateState) Task=`2736b6a8-571d-4933-81c3-027a06a33db4`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:17:41,432::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:17:41,432::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000640246', 'lastCheck': '4.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:17:41,432::task::1185::TaskManager.Task::(prepare) Task=`2736b6a8-571d-4933-81c3-027a06a33db4`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000640246', 'lastCheck': '4.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:17:41,432::task::595::TaskManager.Task::(_updateState) Task=`2736b6a8-571d-4933-81c3-027a06a33db4`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:17:41,432::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:17:41,432::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:17:41,432::task::990::TaskManager.Task::(_decref) Task=`2736b6a8-571d-4933-81c3-027a06a33db4`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:17:41,715::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:17:41,842::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:17:42,063::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306363 s, 33.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:17:44,076::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:17:44,152::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.032421 s, 31.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:17:44,737::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:17:46,165::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:17:46,244::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362049 s, 28.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:17:47,198::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:17:47,206::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000736956 s, 5.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:17:47,756::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:17:48,257::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:17:48,336::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364965 s, 28.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:17:50,349::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 12:17:50,415::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:17:50,417::task::595::TaskManager.Task::(_updateState) Task=`58e5609e-165b-4746-a5bb-3055aa5425c6`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:17:50,418::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:17:50,420::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:17:50,421::task::1185::TaskManager.Task::(prepare) Task=`58e5609e-165b-4746-a5bb-3055aa5425c6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:17:50,422::task::595::TaskManager.Task::(_updateState) Task=`58e5609e-165b-4746-a5bb-3055aa5425c6`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:17:50,422::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:17:50,423::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:17:50,424::task::990::TaskManager.Task::(_decref) Task=`58e5609e-165b-4746-a5bb-3055aa5425c6`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:17:50,441::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0369309 s, 27.7 MB/s\n'; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:17:50,743::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:17:50,744::task::595::TaskManager.Task::(_updateState) Task=`dfe8eeb2-a650-4a7c-af70-4297737a1b68`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:17:50,744::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:17:50,744::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a73c803d-acee-40c2-ae74-002943aaabcc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:17:50,745::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:17:50,745::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:17:50,745::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a73c803d-acee-40c2-ae74-002943aaabcc`::Granted request >Thread-338::DEBUG::2014-05-13 12:17:50,745::task::827::TaskManager.Task::(resourceAcquired) Task=`dfe8eeb2-a650-4a7c-af70-4297737a1b68`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:17:50,746::task::990::TaskManager.Task::(_decref) Task=`dfe8eeb2-a650-4a7c-af70-4297737a1b68`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:17:50,746::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:17:50,746::task::1185::TaskManager.Task::(prepare) Task=`dfe8eeb2-a650-4a7c-af70-4297737a1b68`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:17:50,747::task::595::TaskManager.Task::(_updateState) Task=`dfe8eeb2-a650-4a7c-af70-4297737a1b68`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:17:50,747::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:17:50,747::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:17:50,747::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:17:50,747::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:17:50,748::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:17:50,748::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:17:50,748::task::990::TaskManager.Task::(_decref) Task=`dfe8eeb2-a650-4a7c-af70-4297737a1b68`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:17:50,774::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:17:52,454::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:17:52,547::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0500923 s, 20.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:17:53,793::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:17:54,560::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:17:54,648::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0339948 s, 30.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:17:56,661::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:17:56,735::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303562 s, 33.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:17:56,766::task::595::TaskManager.Task::(_updateState) Task=`e70fe05b-4595-4624-a36e-be07f4ef45ef`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:17:56,766::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:17:56,766::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000736956', 'lastCheck': '9.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:17:56,766::task::1185::TaskManager.Task::(prepare) Task=`e70fe05b-4595-4624-a36e-be07f4ef45ef`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000736956', 'lastCheck': '9.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:17:56,767::task::595::TaskManager.Task::(_updateState) Task=`e70fe05b-4595-4624-a36e-be07f4ef45ef`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:17:56,767::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:17:56,767::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:17:56,767::task::990::TaskManager.Task::(_decref) Task=`e70fe05b-4595-4624-a36e-be07f4ef45ef`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:17:56,811::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:17:57,214::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:17:57,222::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000752523 s, 5.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:17:58,748::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:17:58,828::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362165 s, 28.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:17:59,829::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:18:00,767::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:18:00,768::task::595::TaskManager.Task::(_updateState) Task=`9ba9c4a9-e199-4111-85ea-5dd8c05d6b81`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:18:00,768::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:18:00,768::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:18:00,768::task::1185::TaskManager.Task::(prepare) Task=`9ba9c4a9-e199-4111-85ea-5dd8c05d6b81`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:18:00,769::task::595::TaskManager.Task::(_updateState) Task=`9ba9c4a9-e199-4111-85ea-5dd8c05d6b81`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:18:00,769::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:18:00,769::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:18:00,769::task::990::TaskManager.Task::(_decref) Task=`9ba9c4a9-e199-4111-85ea-5dd8c05d6b81`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:18:00,774::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:18:00,774::task::595::TaskManager.Task::(_updateState) Task=`89c68974-b38d-41cb-8b12-04f63df711bc`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:18:00,775::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:18:00,775::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ffe6203a-c476-4f88-88c2-0f46150f3215`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:18:00,775::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:18:00,775::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:18:00,776::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ffe6203a-c476-4f88-88c2-0f46150f3215`::Granted request >Thread-338::DEBUG::2014-05-13 12:18:00,776::task::827::TaskManager.Task::(resourceAcquired) Task=`89c68974-b38d-41cb-8b12-04f63df711bc`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:18:00,776::task::990::TaskManager.Task::(_decref) Task=`89c68974-b38d-41cb-8b12-04f63df711bc`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:18:00,777::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:18:00,777::task::1185::TaskManager.Task::(prepare) Task=`89c68974-b38d-41cb-8b12-04f63df711bc`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:18:00,777::task::595::TaskManager.Task::(_updateState) Task=`89c68974-b38d-41cb-8b12-04f63df711bc`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:18:00,777::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:18:00,778::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:18:00,778::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:18:00,778::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:18:00,778::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:18:00,778::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:18:00,779::task::990::TaskManager.Task::(_decref) Task=`89c68974-b38d-41cb-8b12-04f63df711bc`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:18:00,839::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:18:00,919::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036256 s, 28.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:18:02,847::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:18:02,932::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:18:03,006::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031181 s, 32.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:18:05,019::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:18:05,109::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0465379 s, 22.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:18:05,866::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:18:07,122::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:18:07,202::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.038425 s, 26.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:18:07,229::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:18:07,237::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000644655 s, 6.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:18:08,886::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:18:09,215::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:18:09,291::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314052 s, 32.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:18:10,797::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6602059] >Thread-13::DEBUG::2014-05-13 12:18:10,798::task::595::TaskManager.Task::(_updateState) Task=`12f8ebc6-8449-42dc-b716-f4feefcc604f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:18:10,798::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:18:10,798::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:18:10,798::task::1185::TaskManager.Task::(prepare) Task=`12f8ebc6-8449-42dc-b716-f4feefcc604f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:18:10,799::task::595::TaskManager.Task::(_updateState) Task=`12f8ebc6-8449-42dc-b716-f4feefcc604f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:18:10,799::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:18:10,799::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:18:10,799::task::990::TaskManager.Task::(_decref) Task=`12f8ebc6-8449-42dc-b716-f4feefcc604f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:18:10,804::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6602059] >Thread-338::DEBUG::2014-05-13 12:18:10,804::task::595::TaskManager.Task::(_updateState) Task=`d1aadd1a-016b-430d-a2bf-c4505c7eee50`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:18:10,804::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:18:10,805::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ecb25545-ab38-477e-b41e-5706ee5288fa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:18:10,805::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:18:10,805::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:18:10,805::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ecb25545-ab38-477e-b41e-5706ee5288fa`::Granted request >Thread-338::DEBUG::2014-05-13 12:18:10,807::task::827::TaskManager.Task::(resourceAcquired) Task=`d1aadd1a-016b-430d-a2bf-c4505c7eee50`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:18:10,807::task::990::TaskManager.Task::(_decref) Task=`d1aadd1a-016b-430d-a2bf-c4505c7eee50`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:18:10,808::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:18:10,808::task::1185::TaskManager.Task::(prepare) Task=`d1aadd1a-016b-430d-a2bf-c4505c7eee50`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:18:10,808::task::595::TaskManager.Task::(_updateState) Task=`d1aadd1a-016b-430d-a2bf-c4505c7eee50`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:18:10,808::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:18:10,808::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:18:10,809::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:18:10,809::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:18:10,809::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:18:10,809::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:18:10,809::task::990::TaskManager.Task::(_decref) Task=`d1aadd1a-016b-430d-a2bf-c4505c7eee50`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:18:11,303::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:18:11,378::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313681 s, 32.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:18:11,900::task::595::TaskManager.Task::(_updateState) Task=`17fd5a01-641a-47e7-ba93-d7df87c9458e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:18:11,901::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:18:11,901::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000644655', 'lastCheck': '4.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:18:11,901::task::1185::TaskManager.Task::(prepare) Task=`17fd5a01-641a-47e7-ba93-d7df87c9458e`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000644655', 'lastCheck': '4.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:18:11,901::task::595::TaskManager.Task::(_updateState) Task=`17fd5a01-641a-47e7-ba93-d7df87c9458e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:18:11,901::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:18:11,902::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:18:11,902::task::990::TaskManager.Task::(_decref) Task=`17fd5a01-641a-47e7-ba93-d7df87c9458e`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:18:11,905::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:18:13,391::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:18:13,473::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373995 s, 27.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:18:14,922::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:18:15,486::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:18:15,567::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374513 s, 27.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:18:17,243::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:18:17,251::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000591154 s, 6.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:18:17,580::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:18:17,658::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358662 s, 28.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:18:17,942::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:18:19,671::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:18:19,767::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0510039 s, 20.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:18:20,827::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [28fd42b9] >Thread-13::DEBUG::2014-05-13 12:18:20,828::task::595::TaskManager.Task::(_updateState) Task=`ed135b34-639a-4e41-aaa6-e00e49c1e6f4`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:18:20,828::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:18:20,828::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:18:20,828::task::1185::TaskManager.Task::(prepare) Task=`ed135b34-639a-4e41-aaa6-e00e49c1e6f4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:18:20,829::task::595::TaskManager.Task::(_updateState) Task=`ed135b34-639a-4e41-aaa6-e00e49c1e6f4`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:18:20,829::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:18:20,829::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:18:20,829::task::990::TaskManager.Task::(_decref) Task=`ed135b34-639a-4e41-aaa6-e00e49c1e6f4`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:18:20,834::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [28fd42b9] >Thread-338::DEBUG::2014-05-13 12:18:20,834::task::595::TaskManager.Task::(_updateState) Task=`40a756ea-184f-4a8a-974d-0b0ca30ff361`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:18:20,834::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:18:20,835::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c2a13dfd-c2cc-4c22-8940-14d973c2d993`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:18:20,835::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:18:20,835::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:18:20,835::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c2a13dfd-c2cc-4c22-8940-14d973c2d993`::Granted request >Thread-338::DEBUG::2014-05-13 12:18:20,836::task::827::TaskManager.Task::(resourceAcquired) Task=`40a756ea-184f-4a8a-974d-0b0ca30ff361`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:18:20,836::task::990::TaskManager.Task::(_decref) Task=`40a756ea-184f-4a8a-974d-0b0ca30ff361`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:18:20,837::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:18:20,837::task::1185::TaskManager.Task::(prepare) Task=`40a756ea-184f-4a8a-974d-0b0ca30ff361`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:18:20,837::task::595::TaskManager.Task::(_updateState) Task=`40a756ea-184f-4a8a-974d-0b0ca30ff361`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:18:20,837::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:18:20,837::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:18:20,838::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:18:20,838::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:18:20,838::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:18:20,838::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:18:20,838::task::990::TaskManager.Task::(_decref) Task=`40a756ea-184f-4a8a-974d-0b0ca30ff361`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:18:20,959::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:18:21,780::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:18:21,859::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355814 s, 28.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:18:23,872::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:18:23,945::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306838 s, 33.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:18:23,978::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:18:25,959::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:18:26,033::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311392 s, 32.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:18:26,997::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:18:27,045::task::595::TaskManager.Task::(_updateState) Task=`b3e313e2-bdf8-4ac3-86c8-d4edcf5d1843`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:18:27,045::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:18:27,045::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000591154', 'lastCheck': '9.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:18:27,046::task::1185::TaskManager.Task::(prepare) Task=`b3e313e2-bdf8-4ac3-86c8-d4edcf5d1843`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000591154', 'lastCheck': '9.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:18:27,046::task::595::TaskManager.Task::(_updateState) Task=`b3e313e2-bdf8-4ac3-86c8-d4edcf5d1843`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:18:27,046::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:18:27,046::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:18:27,046::task::990::TaskManager.Task::(_decref) Task=`b3e313e2-bdf8-4ac3-86c8-d4edcf5d1843`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 12:18:27,258::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:18:27,267::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00159591 s, 2.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:18:28,046::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:18:28,127::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0368281 s, 27.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:18:30,015::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:18:30,140::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:18:30,220::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037225 s, 27.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:18:30,862::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:18:30,862::task::595::TaskManager.Task::(_updateState) Task=`6bdef151-4735-4450-9399-58cdc9473a81`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:18:30,862::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:18:30,863::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:18:30,863::task::1185::TaskManager.Task::(prepare) Task=`6bdef151-4735-4450-9399-58cdc9473a81`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:18:30,863::task::595::TaskManager.Task::(_updateState) Task=`6bdef151-4735-4450-9399-58cdc9473a81`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:18:30,863::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:18:30,863::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:18:30,864::task::990::TaskManager.Task::(_decref) Task=`6bdef151-4735-4450-9399-58cdc9473a81`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:18:30,869::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:18:30,869::task::595::TaskManager.Task::(_updateState) Task=`d2d445d1-9820-49cf-9869-696245c43dd8`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:18:30,870::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:18:30,870::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`67d713f7-f6b1-4a63-9c94-f4a551f3bdaf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:18:30,870::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:18:30,871::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:18:30,871::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`67d713f7-f6b1-4a63-9c94-f4a551f3bdaf`::Granted request >Thread-338::DEBUG::2014-05-13 12:18:30,871::task::827::TaskManager.Task::(resourceAcquired) Task=`d2d445d1-9820-49cf-9869-696245c43dd8`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:18:30,871::task::990::TaskManager.Task::(_decref) Task=`d2d445d1-9820-49cf-9869-696245c43dd8`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:18:30,872::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:18:30,872::task::1185::TaskManager.Task::(prepare) Task=`d2d445d1-9820-49cf-9869-696245c43dd8`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:18:30,872::task::595::TaskManager.Task::(_updateState) Task=`d2d445d1-9820-49cf-9869-696245c43dd8`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:18:30,873::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:18:30,873::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:18:30,873::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:18:30,873::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:18:30,873::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:18:30,873::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:18:30,874::task::990::TaskManager.Task::(_decref) Task=`d2d445d1-9820-49cf-9869-696245c43dd8`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:18:32,233::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:18:32,309::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317074 s, 32.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:18:33,035::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:18:34,322::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:18:34,401::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374938 s, 27.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:18:36,060::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:18:36,414::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:18:36,794::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.336257 s, 3.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:18:37,274::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:18:37,284::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00195806 s, 2.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:18:38,808::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:18:38,883::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0329333 s, 31.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:18:39,079::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:18:40,894::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:18:40,895::task::595::TaskManager.Task::(_updateState) Task=`abfc5c8b-ccab-4825-bd53-8bdc7eb90d58`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:18:40,895::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:18:40,895::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:18:40,896::task::1185::TaskManager.Task::(prepare) Task=`abfc5c8b-ccab-4825-bd53-8bdc7eb90d58`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Dummy-337::DEBUG::2014-05-13 12:18:40,896::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 12:18:40,896::task::595::TaskManager.Task::(_updateState) Task=`abfc5c8b-ccab-4825-bd53-8bdc7eb90d58`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:18:40,903::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:18:40,903::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:18:40,904::task::990::TaskManager.Task::(_decref) Task=`abfc5c8b-ccab-4825-bd53-8bdc7eb90d58`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:18:40,912::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:18:40,912::task::595::TaskManager.Task::(_updateState) Task=`89f01323-5ecc-437a-b1b5-26082d62b0b0`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:18:40,912::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:18:40,913::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3674c111-a2d9-458e-ae54-3e6f9a802e62`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:18:40,913::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:18:40,914::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:18:40,914::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3674c111-a2d9-458e-ae54-3e6f9a802e62`::Granted request >Thread-338::DEBUG::2014-05-13 12:18:40,914::task::827::TaskManager.Task::(resourceAcquired) Task=`89f01323-5ecc-437a-b1b5-26082d62b0b0`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:18:40,915::task::990::TaskManager.Task::(_decref) Task=`89f01323-5ecc-437a-b1b5-26082d62b0b0`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:18:40,916::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:18:40,916::task::1185::TaskManager.Task::(prepare) Task=`89f01323-5ecc-437a-b1b5-26082d62b0b0`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:18:40,916::task::595::TaskManager.Task::(_updateState) Task=`89f01323-5ecc-437a-b1b5-26082d62b0b0`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:18:40,916::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:18:40,917::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:18:40,917::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:18:40,917::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:18:40,918::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:18:40,918::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:18:40,918::task::990::TaskManager.Task::(_decref) Task=`89f01323-5ecc-437a-b1b5-26082d62b0b0`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:18:40,978::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0380214 s, 26.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:18:42,097::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:18:42,179::task::595::TaskManager.Task::(_updateState) Task=`d83862e0-5cda-4be4-8a7c-500b21eeba21`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:18:42,179::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:18:42,179::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00195806', 'lastCheck': '4.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:18:42,179::task::1185::TaskManager.Task::(prepare) Task=`d83862e0-5cda-4be4-8a7c-500b21eeba21`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00195806', 'lastCheck': '4.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:18:42,179::task::595::TaskManager.Task::(_updateState) Task=`d83862e0-5cda-4be4-8a7c-500b21eeba21`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:18:42,179::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:18:42,180::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:18:42,180::task::990::TaskManager.Task::(_decref) Task=`d83862e0-5cda-4be4-8a7c-500b21eeba21`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:18:42,991::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:18:43,073::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0376464 s, 27.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:18:45,085::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:18:45,119::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:18:45,169::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0400246 s, 25.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:18:47,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 12:18:47,290::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:18:47,597::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.292815 s, 14.0 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:18:47,681::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.457338 s, 2.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:18:48,138::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:18:49,694::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:18:49,774::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0375492 s, 27.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:18:50,941::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:18:50,941::task::595::TaskManager.Task::(_updateState) Task=`8755344b-0fdf-4c9d-a6bf-b45b24deff18`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:18:50,941::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:18:50,942::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:18:50,942::task::1185::TaskManager.Task::(prepare) Task=`8755344b-0fdf-4c9d-a6bf-b45b24deff18`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:18:50,942::task::595::TaskManager.Task::(_updateState) Task=`8755344b-0fdf-4c9d-a6bf-b45b24deff18`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:18:50,942::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:18:50,943::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:18:50,943::task::990::TaskManager.Task::(_decref) Task=`8755344b-0fdf-4c9d-a6bf-b45b24deff18`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:18:50,947::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:18:50,947::task::595::TaskManager.Task::(_updateState) Task=`e657c75c-1c19-48b7-b9c0-62d3154e4730`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:18:50,948::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:18:50,948::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`65fadecc-ce89-4ca5-873d-ff462bc272c9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:18:50,948::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:18:50,949::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:18:50,949::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`65fadecc-ce89-4ca5-873d-ff462bc272c9`::Granted request >Thread-338::DEBUG::2014-05-13 12:18:50,949::task::827::TaskManager.Task::(resourceAcquired) Task=`e657c75c-1c19-48b7-b9c0-62d3154e4730`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:18:50,949::task::990::TaskManager.Task::(_decref) Task=`e657c75c-1c19-48b7-b9c0-62d3154e4730`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:18:50,950::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:18:50,950::task::1185::TaskManager.Task::(prepare) Task=`e657c75c-1c19-48b7-b9c0-62d3154e4730`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:18:50,950::task::595::TaskManager.Task::(_updateState) Task=`e657c75c-1c19-48b7-b9c0-62d3154e4730`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:18:50,951::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:18:50,951::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:18:50,951::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:18:50,951::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:18:50,951::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:18:50,951::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:18:50,952::task::990::TaskManager.Task::(_decref) Task=`e657c75c-1c19-48b7-b9c0-62d3154e4730`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:18:51,155::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:18:51,787::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:18:51,867::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0378801 s, 27.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:18:53,880::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:18:53,955::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0339063 s, 30.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:18:54,173::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:18:55,968::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:18:56,045::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0337776 s, 30.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:18:57,198::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:18:57,315::task::595::TaskManager.Task::(_updateState) Task=`8d16ac69-313c-4679-960e-14e9dc03ae68`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:18:57,315::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:18:57,315::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.292815', 'lastCheck': '9.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:18:57,315::task::1185::TaskManager.Task::(prepare) Task=`8d16ac69-313c-4679-960e-14e9dc03ae68`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.292815', 'lastCheck': '9.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:18:57,316::task::595::TaskManager.Task::(_updateState) Task=`8d16ac69-313c-4679-960e-14e9dc03ae68`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:18:57,316::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:18:57,316::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:18:57,316::task::990::TaskManager.Task::(_decref) Task=`8d16ac69-313c-4679-960e-14e9dc03ae68`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 12:18:57,624::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:18:57,632::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000688006 s, 6.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:18:58,058::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:18:58,139::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0386632 s, 26.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:19:00,153::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:19:00,229::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:19:00,249::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0490689 s, 20.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:19:00,970::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:19:00,970::task::595::TaskManager.Task::(_updateState) Task=`9be2da09-6195-4e15-9868-788fa1f693d4`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:19:00,971::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:19:00,971::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:19:00,971::task::1185::TaskManager.Task::(prepare) Task=`9be2da09-6195-4e15-9868-788fa1f693d4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:19:00,971::task::595::TaskManager.Task::(_updateState) Task=`9be2da09-6195-4e15-9868-788fa1f693d4`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:19:00,972::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:19:00,972::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:19:00,972::task::990::TaskManager.Task::(_decref) Task=`9be2da09-6195-4e15-9868-788fa1f693d4`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:19:00,976::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:19:00,976::task::595::TaskManager.Task::(_updateState) Task=`fbc0236c-203e-45b0-bf4d-9623c11b6f72`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:19:00,976::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:19:00,977::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6d4de13e-3d92-4256-9192-e5d55f44e36a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:19:00,977::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:19:00,977::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:19:00,977::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6d4de13e-3d92-4256-9192-e5d55f44e36a`::Granted request >Thread-338::DEBUG::2014-05-13 12:19:00,978::task::827::TaskManager.Task::(resourceAcquired) Task=`fbc0236c-203e-45b0-bf4d-9623c11b6f72`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:19:00,978::task::990::TaskManager.Task::(_decref) Task=`fbc0236c-203e-45b0-bf4d-9623c11b6f72`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:19:00,979::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:19:00,979::task::1185::TaskManager.Task::(prepare) Task=`fbc0236c-203e-45b0-bf4d-9623c11b6f72`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:19:00,979::task::595::TaskManager.Task::(_updateState) Task=`fbc0236c-203e-45b0-bf4d-9623c11b6f72`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:19:00,979::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:19:00,980::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:19:00,980::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:19:00,980::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:19:00,980::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:19:00,980::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:19:00,981::task::990::TaskManager.Task::(_decref) Task=`fbc0236c-203e-45b0-bf4d-9623c11b6f72`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:19:02,261::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:19:02,347::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0436601 s, 23.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:19:03,250::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:19:04,360::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:19:04,440::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384065 s, 26.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:19:06,275::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:19:06,453::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:19:06,535::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0396293 s, 25.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:19:07,639::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:19:07,647::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000677763 s, 6.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:19:08,548::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:19:08,624::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0337415 s, 30.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:19:09,293::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:19:10,637::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:19:10,717::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0386647 s, 26.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:19:11,001::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [30b3ccf] >Thread-13::DEBUG::2014-05-13 12:19:11,001::task::595::TaskManager.Task::(_updateState) Task=`980d9878-3025-4012-96c0-977d07521be6`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:19:11,001::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:19:11,002::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:19:11,002::task::1185::TaskManager.Task::(prepare) Task=`980d9878-3025-4012-96c0-977d07521be6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:19:11,002::task::595::TaskManager.Task::(_updateState) Task=`980d9878-3025-4012-96c0-977d07521be6`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:19:11,002::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:19:11,002::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:19:11,002::task::990::TaskManager.Task::(_decref) Task=`980d9878-3025-4012-96c0-977d07521be6`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:19:11,007::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [30b3ccf] >Thread-338::DEBUG::2014-05-13 12:19:11,007::task::595::TaskManager.Task::(_updateState) Task=`c7c4c2ed-822a-400c-8b5f-96fc9073ca31`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:19:11,007::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:19:11,008::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5dfce297-6de7-48d2-8d0c-ec1562192264`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:19:11,008::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:19:11,008::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:19:11,008::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5dfce297-6de7-48d2-8d0c-ec1562192264`::Granted request >Thread-338::DEBUG::2014-05-13 12:19:11,009::task::827::TaskManager.Task::(resourceAcquired) Task=`c7c4c2ed-822a-400c-8b5f-96fc9073ca31`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:19:11,009::task::990::TaskManager.Task::(_decref) Task=`c7c4c2ed-822a-400c-8b5f-96fc9073ca31`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:19:11,009::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:19:11,009::task::1185::TaskManager.Task::(prepare) Task=`c7c4c2ed-822a-400c-8b5f-96fc9073ca31`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:19:11,010::task::595::TaskManager.Task::(_updateState) Task=`c7c4c2ed-822a-400c-8b5f-96fc9073ca31`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:19:11,010::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:19:11,010::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:19:11,010::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:19:11,010::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:19:11,010::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:19:11,011::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:19:11,011::task::990::TaskManager.Task::(_decref) Task=`c7c4c2ed-822a-400c-8b5f-96fc9073ca31`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:19:12,315::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:19:12,445::task::595::TaskManager.Task::(_updateState) Task=`7a9e49be-5658-4356-a55b-2c3a6eb136aa`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:19:12,445::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:19:12,445::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000677763', 'lastCheck': '4.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:19:12,445::task::1185::TaskManager.Task::(prepare) Task=`7a9e49be-5658-4356-a55b-2c3a6eb136aa`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000677763', 'lastCheck': '4.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:19:12,446::task::595::TaskManager.Task::(_updateState) Task=`7a9e49be-5658-4356-a55b-2c3a6eb136aa`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:19:12,446::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:19:12,446::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:19:12,446::task::990::TaskManager.Task::(_decref) Task=`7a9e49be-5658-4356-a55b-2c3a6eb136aa`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:19:12,730::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:19:12,804::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0326486 s, 31.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:19:14,817::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:19:14,908::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0491003 s, 20.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:19:15,332::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:19:16,921::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:19:16,995::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0304641 s, 33.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:19:17,654::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:19:17,662::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000615692 s, 6.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:19:18,352::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:19:19,008::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:19:19,087::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036483 s, 28.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:19:21,031::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:19:21,031::task::595::TaskManager.Task::(_updateState) Task=`29c0f278-3571-45e5-ab72-233ee136a901`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:19:21,032::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:19:21,032::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:19:21,032::task::1185::TaskManager.Task::(prepare) Task=`29c0f278-3571-45e5-ab72-233ee136a901`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:19:21,032::task::595::TaskManager.Task::(_updateState) Task=`29c0f278-3571-45e5-ab72-233ee136a901`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:19:21,032::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:19:21,033::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:19:21,033::task::990::TaskManager.Task::(_decref) Task=`29c0f278-3571-45e5-ab72-233ee136a901`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:19:21,037::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:19:21,038::task::595::TaskManager.Task::(_updateState) Task=`d3cbd698-c483-470b-b106-83336aa68c2b`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:19:21,038::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:19:21,038::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9468ef3e-fa5e-4046-8250-780f36a15c1d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:19:21,039::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:19:21,039::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:19:21,039::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9468ef3e-fa5e-4046-8250-780f36a15c1d`::Granted request >Thread-338::DEBUG::2014-05-13 12:19:21,040::task::827::TaskManager.Task::(resourceAcquired) Task=`d3cbd698-c483-470b-b106-83336aa68c2b`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:19:21,040::task::990::TaskManager.Task::(_decref) Task=`d3cbd698-c483-470b-b106-83336aa68c2b`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:19:21,041::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:19:21,041::task::1185::TaskManager.Task::(prepare) Task=`d3cbd698-c483-470b-b106-83336aa68c2b`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:19:21,042::task::595::TaskManager.Task::(_updateState) Task=`d3cbd698-c483-470b-b106-83336aa68c2b`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:19:21,042::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:19:21,042::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:19:21,042::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:19:21,042::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:19:21,043::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:19:21,043::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:19:21,043::task::990::TaskManager.Task::(_decref) Task=`d3cbd698-c483-470b-b106-83336aa68c2b`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:19:21,100::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:19:21,181::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381396 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:19:21,372::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:19:23,194::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:19:23,269::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0332224 s, 30.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:19:24,389::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:19:25,282::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:19:25,363::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384008 s, 26.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:19:27,376::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:19:27,408::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:19:27,575::task::595::TaskManager.Task::(_updateState) Task=`387dfc8b-7a34-4f3c-aef8-2bc417835154`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:19:27,576::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:19:27,576::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000615692', 'lastCheck': '9.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:19:27,576::task::1185::TaskManager.Task::(prepare) Task=`387dfc8b-7a34-4f3c-aef8-2bc417835154`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000615692', 'lastCheck': '9.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:19:27,576::task::595::TaskManager.Task::(_updateState) Task=`387dfc8b-7a34-4f3c-aef8-2bc417835154`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:19:27,576::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:19:27,576::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:19:27,577::task::990::TaskManager.Task::(_decref) Task=`387dfc8b-7a34-4f3c-aef8-2bc417835154`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 12:19:27,669::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:19:27,975::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.291392 s, 14.1 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:19:28,061::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.642566 s, 1.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:19:30,073::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:19:30,149::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317703 s, 32.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:19:30,429::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:19:31,061::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4478bd69] >Thread-13::DEBUG::2014-05-13 12:19:31,061::task::595::TaskManager.Task::(_updateState) Task=`6f5064e8-3ec5-479b-acda-963ae1d0d020`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:19:31,062::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:19:31,062::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:19:31,062::task::1185::TaskManager.Task::(prepare) Task=`6f5064e8-3ec5-479b-acda-963ae1d0d020`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:19:31,062::task::595::TaskManager.Task::(_updateState) Task=`6f5064e8-3ec5-479b-acda-963ae1d0d020`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:19:31,063::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:19:31,063::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:19:31,063::task::990::TaskManager.Task::(_decref) Task=`6f5064e8-3ec5-479b-acda-963ae1d0d020`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:19:31,067::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4478bd69] >Thread-338::DEBUG::2014-05-13 12:19:31,067::task::595::TaskManager.Task::(_updateState) Task=`8c149be2-6fcf-4c9d-98e9-9301fe45a4e4`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:19:31,068::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:19:31,068::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1867101d-3ce5-422e-9ce6-e9762cedce06`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:19:31,068::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:19:31,068::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:19:31,069::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1867101d-3ce5-422e-9ce6-e9762cedce06`::Granted request >Thread-338::DEBUG::2014-05-13 12:19:31,069::task::827::TaskManager.Task::(resourceAcquired) Task=`8c149be2-6fcf-4c9d-98e9-9301fe45a4e4`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:19:31,069::task::990::TaskManager.Task::(_decref) Task=`8c149be2-6fcf-4c9d-98e9-9301fe45a4e4`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:19:31,070::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:19:31,070::task::1185::TaskManager.Task::(prepare) Task=`8c149be2-6fcf-4c9d-98e9-9301fe45a4e4`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:19:31,070::task::595::TaskManager.Task::(_updateState) Task=`8c149be2-6fcf-4c9d-98e9-9301fe45a4e4`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:19:31,070::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:19:31,071::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:19:31,071::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:19:31,071::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:19:31,071::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:19:31,071::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:19:31,072::task::990::TaskManager.Task::(_decref) Task=`8c149be2-6fcf-4c9d-98e9-9301fe45a4e4`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:19:32,162::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:19:32,235::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303827 s, 33.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:19:33,452::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:19:34,248::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:19:34,330::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381485 s, 26.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:19:36,343::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:19:36,426::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0389347 s, 26.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:19:36,469::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:19:38,001::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:19:38,009::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000628123 s, 6.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:19:38,439::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:19:38,523::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0415451 s, 24.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:19:39,492::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:19:40,536::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:19:40,631::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.053027 s, 19.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:19:41,090::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:19:41,090::task::595::TaskManager.Task::(_updateState) Task=`991447d4-1df9-4de2-8319-6b41135d7cd4`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:19:41,091::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:19:41,091::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:19:41,091::task::1185::TaskManager.Task::(prepare) Task=`991447d4-1df9-4de2-8319-6b41135d7cd4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:19:41,091::task::595::TaskManager.Task::(_updateState) Task=`991447d4-1df9-4de2-8319-6b41135d7cd4`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:19:41,091::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:19:41,091::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:19:41,092::task::990::TaskManager.Task::(_decref) Task=`991447d4-1df9-4de2-8319-6b41135d7cd4`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:19:41,096::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:19:41,097::task::595::TaskManager.Task::(_updateState) Task=`7bd087a6-a922-49c9-8afa-d144c8e33a43`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:19:41,097::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:19:41,097::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4bfee2ff-dd11-40b7-896c-07a5fcb5e84f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:19:41,097::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:19:41,097::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:19:41,098::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4bfee2ff-dd11-40b7-896c-07a5fcb5e84f`::Granted request >Thread-338::DEBUG::2014-05-13 12:19:41,098::task::827::TaskManager.Task::(resourceAcquired) Task=`7bd087a6-a922-49c9-8afa-d144c8e33a43`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:19:41,098::task::990::TaskManager.Task::(_decref) Task=`7bd087a6-a922-49c9-8afa-d144c8e33a43`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:19:41,099::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:19:41,099::task::1185::TaskManager.Task::(prepare) Task=`7bd087a6-a922-49c9-8afa-d144c8e33a43`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:19:41,099::task::595::TaskManager.Task::(_updateState) Task=`7bd087a6-a922-49c9-8afa-d144c8e33a43`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:19:41,099::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:19:41,099::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:19:41,100::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:19:41,100::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:19:41,100::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:19:41,100::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:19:41,100::task::990::TaskManager.Task::(_decref) Task=`7bd087a6-a922-49c9-8afa-d144c8e33a43`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:19:42,511::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:19:42,644::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 12:19:42,723::task::595::TaskManager.Task::(_updateState) Task=`d3457abd-b8a6-45be-9168-68af3862b102`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:19:42,725::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:19:42,727::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000628123', 'lastCheck': '4.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:19:42,728::task::1185::TaskManager.Task::(prepare) Task=`d3457abd-b8a6-45be-9168-68af3862b102`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000628123', 'lastCheck': '4.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:19:42,730::task::595::TaskManager.Task::(_updateState) Task=`d3457abd-b8a6-45be-9168-68af3862b102`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:19:42,731::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:19:42,732::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:19:42,734::task::990::TaskManager.Task::(_decref) Task=`d3457abd-b8a6-45be-9168-68af3862b102`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:19:42,760::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0346687 s, 29.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:19:44,775::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:19:44,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0328424 s, 31.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:19:45,529::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:19:46,864::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:19:46,956::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0367901 s, 27.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:19:48,016::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:19:48,024::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000656523 s, 6.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:19:48,547::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:19:48,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:19:49,044::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362032 s, 28.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:19:51,057::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 12:19:51,120::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [28fd42b9] >Thread-13::DEBUG::2014-05-13 12:19:51,122::task::595::TaskManager.Task::(_updateState) Task=`7e242429-c345-4268-b096-0f3bc0c2746f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:19:51,123::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:19:51,125::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:19:51,125::task::1185::TaskManager.Task::(prepare) Task=`7e242429-c345-4268-b096-0f3bc0c2746f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:19:51,126::task::595::TaskManager.Task::(_updateState) Task=`7e242429-c345-4268-b096-0f3bc0c2746f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:19:51,127::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:19:51,128::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:19:51,128::task::990::TaskManager.Task::(_decref) Task=`7e242429-c345-4268-b096-0f3bc0c2746f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:19:51,136::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [28fd42b9] >Thread-338::DEBUG::2014-05-13 12:19:51,138::task::595::TaskManager.Task::(_updateState) Task=`6442f2e5-ba3c-4e64-8657-419fe5e1abe7`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:19:51,139::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:19:51,141::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d9fb5868-8c71-4c3a-a876-886d4fa0762b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:19:51,142::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:19:51,143::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:19:51,144::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d9fb5868-8c71-4c3a-a876-886d4fa0762b`::Granted request >Thread-338::DEBUG::2014-05-13 12:19:51,146::task::827::TaskManager.Task::(resourceAcquired) Task=`6442f2e5-ba3c-4e64-8657-419fe5e1abe7`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:19:51,147::task::990::TaskManager.Task::(_decref) Task=`6442f2e5-ba3c-4e64-8657-419fe5e1abe7`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:19:51,151::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:19:51,152::task::1185::TaskManager.Task::(prepare) Task=`6442f2e5-ba3c-4e64-8657-419fe5e1abe7`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:19:51,153::task::595::TaskManager.Task::(_updateState) Task=`6442f2e5-ba3c-4e64-8657-419fe5e1abe7`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:19:51,154::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:19:51,155::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:19:51,156::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:19:51,158::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:19:51,158::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:19:51,159::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:19:51,161::task::990::TaskManager.Task::(_decref) Task=`6442f2e5-ba3c-4e64-8657-419fe5e1abe7`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:19:51,177::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0370297 s, 27.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:19:51,566::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:19:53,189::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:19:53,264::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318779 s, 32.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:19:54,585::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:19:55,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:19:55,696::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.374187 s, 2.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:19:57,603::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:19:57,708::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:19:57,784::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319491 s, 32.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:19:57,888::task::595::TaskManager.Task::(_updateState) Task=`17df9fcf-358f-4955-95cd-b00fe72d0552`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:19:57,888::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:19:57,888::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000656523', 'lastCheck': '9.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:19:57,888::task::1185::TaskManager.Task::(prepare) Task=`17df9fcf-358f-4955-95cd-b00fe72d0552`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000656523', 'lastCheck': '9.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:19:57,889::task::595::TaskManager.Task::(_updateState) Task=`17df9fcf-358f-4955-95cd-b00fe72d0552`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:19:57,889::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:19:57,889::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:19:57,889::task::990::TaskManager.Task::(_decref) Task=`17df9fcf-358f-4955-95cd-b00fe72d0552`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 12:19:58,063::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:19:58,071::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000744295 s, 5.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:19:59,797::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:19:59,872::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031863 s, 32.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:20:00,622::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:20:01,195::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:20:01,196::task::595::TaskManager.Task::(_updateState) Task=`bd9b4fab-b815-4aa8-bdc5-e3c1c88cc08a`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:20:01,196::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:20:01,196::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:20:01,196::task::1185::TaskManager.Task::(prepare) Task=`bd9b4fab-b815-4aa8-bdc5-e3c1c88cc08a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:20:01,197::task::595::TaskManager.Task::(_updateState) Task=`bd9b4fab-b815-4aa8-bdc5-e3c1c88cc08a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:20:01,197::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:20:01,197::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:20:01,197::task::990::TaskManager.Task::(_decref) Task=`bd9b4fab-b815-4aa8-bdc5-e3c1c88cc08a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:20:01,202::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:20:01,203::task::595::TaskManager.Task::(_updateState) Task=`6a825430-136e-48c8-a025-7e15b80655ed`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:20:01,203::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:20:01,203::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9f417e35-0b26-485c-b61d-9d276444db4c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:20:01,204::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:20:01,204::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:20:01,204::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9f417e35-0b26-485c-b61d-9d276444db4c`::Granted request >Thread-338::DEBUG::2014-05-13 12:20:01,204::task::827::TaskManager.Task::(resourceAcquired) Task=`6a825430-136e-48c8-a025-7e15b80655ed`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:20:01,205::task::990::TaskManager.Task::(_decref) Task=`6a825430-136e-48c8-a025-7e15b80655ed`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:20:01,205::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:20:01,205::task::1185::TaskManager.Task::(prepare) Task=`6a825430-136e-48c8-a025-7e15b80655ed`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:20:01,206::task::595::TaskManager.Task::(_updateState) Task=`6a825430-136e-48c8-a025-7e15b80655ed`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:20:01,206::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:20:01,206::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:20:01,206::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:20:01,206::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:20:01,207::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:20:01,207::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:20:01,207::task::990::TaskManager.Task::(_decref) Task=`6a825430-136e-48c8-a025-7e15b80655ed`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:20:01,885::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:01,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361263 s, 28.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:20:03,641::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:20:03,973::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:04,047::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320304 s, 32.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:20:06,061::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:06,140::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361718 s, 28.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:20:06,659::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:20:08,075::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 41830dad-f012-4b4d-b636-863f66d5ca4b >Thread-322::DEBUG::2014-05-13 12:20:08,076::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-322::DEBUG::2014-05-13 12:20:08,078::blockSD::336::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:20:08,086::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000540582 s, 3.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:20:08,086::misc::287::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000540582 s, 3.8 MB/s'], size: 2048 >Thread-322::DEBUG::2014-05-13 12:20:08,086::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-322::WARNING::2014-05-13 12:20:08,086::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-322::DEBUG::2014-05-13 12:20:08,087::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-322::DEBUG::2014-05-13 12:20:08,087::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:20:08,087::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:20:08,088::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:20:08,088::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:20:08,088::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:20:08,089::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:08,153::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:08,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0791348 s, 12.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:20:08,362::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:20:08,364::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:20:08,364::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'LOCKPOLICY=', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', 'DESCRIPTION=ISCSI', 'LOGBLKSIZE=512', 'TYPE=ISCSI', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'PHYBLKSIZE=512', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', 'CLASS=Data', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'POOL_DESCRIPTION=DC_2', 'POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'MASTER_VERSION=1', 'ROLE=Master', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', '_SHA_CKSUM=28bb2d7cc60e0d779eb69fe93d050c16c769f069', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0'] >Thread-322::INFO::2014-05-13 12:20:08,365::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_imageNS already registered >Thread-322::INFO::2014-05-13 12:20:08,365::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_volumeNS already registered >Thread-322::INFO::2014-05-13 12:20:08,366::blockSD::452::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_lvmActivationNS already registered >Thread-322::DEBUG::2014-05-13 12:20:08,366::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 12:20:08,576::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:20:08,579::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:20:08,587::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000641824 s, 6.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:20:09,681::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:20:10,291::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:10,371::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0367761 s, 27.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:20:11,225::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:20:11,226::task::595::TaskManager.Task::(_updateState) Task=`ea9ecb2b-8692-433d-8aab-d5204537950e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:20:11,226::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:20:11,227::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:20:11,227::task::1185::TaskManager.Task::(prepare) Task=`ea9ecb2b-8692-433d-8aab-d5204537950e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:20:11,227::task::595::TaskManager.Task::(_updateState) Task=`ea9ecb2b-8692-433d-8aab-d5204537950e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:20:11,227::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:20:11,227::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:20:11,227::task::990::TaskManager.Task::(_decref) Task=`ea9ecb2b-8692-433d-8aab-d5204537950e`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:20:11,232::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:20:11,233::task::595::TaskManager.Task::(_updateState) Task=`90c679cc-6a92-4ec3-8bbc-03275dc089c0`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:20:11,233::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:20:11,233::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`23764057-12c3-4b21-a8e1-79b5a283cfa6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:20:11,233::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:20:11,234::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:20:11,234::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`23764057-12c3-4b21-a8e1-79b5a283cfa6`::Granted request >Thread-338::DEBUG::2014-05-13 12:20:11,234::task::827::TaskManager.Task::(resourceAcquired) Task=`90c679cc-6a92-4ec3-8bbc-03275dc089c0`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:20:11,234::task::990::TaskManager.Task::(_decref) Task=`90c679cc-6a92-4ec3-8bbc-03275dc089c0`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:20:11,235::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:20:11,235::task::1185::TaskManager.Task::(prepare) Task=`90c679cc-6a92-4ec3-8bbc-03275dc089c0`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:20:11,235::task::595::TaskManager.Task::(_updateState) Task=`90c679cc-6a92-4ec3-8bbc-03275dc089c0`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:20:11,235::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:20:11,235::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:20:11,236::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:20:11,236::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:20:11,236::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:20:11,236::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:20:11,236::task::990::TaskManager.Task::(_decref) Task=`90c679cc-6a92-4ec3-8bbc-03275dc089c0`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:20:12,384::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:12,457::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310045 s, 33.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:20:12,701::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:20:13,025::task::595::TaskManager.Task::(_updateState) Task=`2e72ccf5-836e-49ae-9a3a-61a106100204`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:20:13,026::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:20:13,026::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000641824', 'lastCheck': '4.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:20:13,026::task::1185::TaskManager.Task::(prepare) Task=`2e72ccf5-836e-49ae-9a3a-61a106100204`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000641824', 'lastCheck': '4.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:20:13,026::task::595::TaskManager.Task::(_updateState) Task=`2e72ccf5-836e-49ae-9a3a-61a106100204`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:20:13,026::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:20:13,026::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:20:13,027::task::990::TaskManager.Task::(_decref) Task=`2e72ccf5-836e-49ae-9a3a-61a106100204`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:20:14,471::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:14,546::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0330028 s, 31.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:20:15,719::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:20:16,559::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:16,640::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0376494 s, 27.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:20:18,594::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:20:18,602::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00064667 s, 6.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:20:18,653::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:18,733::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036739 s, 27.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:20:18,738::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:20:20,749::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:20,841::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.049772 s, 20.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:20:21,254::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2555e502] >Thread-13::DEBUG::2014-05-13 12:20:21,255::task::595::TaskManager.Task::(_updateState) Task=`95c0497b-71b6-46ab-a64f-ba8a13d4c67f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:20:21,255::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:20:21,255::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:20:21,255::task::1185::TaskManager.Task::(prepare) Task=`95c0497b-71b6-46ab-a64f-ba8a13d4c67f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:20:21,255::task::595::TaskManager.Task::(_updateState) Task=`95c0497b-71b6-46ab-a64f-ba8a13d4c67f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:20:21,256::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:20:21,256::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:20:21,256::task::990::TaskManager.Task::(_decref) Task=`95c0497b-71b6-46ab-a64f-ba8a13d4c67f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:20:21,261::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2555e502] >Thread-338::DEBUG::2014-05-13 12:20:21,261::task::595::TaskManager.Task::(_updateState) Task=`a82f1638-12a8-4405-8e83-d7a68232e107`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:20:21,261::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:20:21,262::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`af0984aa-0c35-4249-b91b-23b95bcd0a28`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:20:21,262::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:20:21,262::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:20:21,263::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`af0984aa-0c35-4249-b91b-23b95bcd0a28`::Granted request >Thread-338::DEBUG::2014-05-13 12:20:21,263::task::827::TaskManager.Task::(resourceAcquired) Task=`a82f1638-12a8-4405-8e83-d7a68232e107`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:20:21,263::task::990::TaskManager.Task::(_decref) Task=`a82f1638-12a8-4405-8e83-d7a68232e107`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:20:21,264::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:20:21,264::task::1185::TaskManager.Task::(prepare) Task=`a82f1638-12a8-4405-8e83-d7a68232e107`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:20:21,264::task::595::TaskManager.Task::(_updateState) Task=`a82f1638-12a8-4405-8e83-d7a68232e107`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:20:21,264::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:20:21,265::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:20:21,265::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:20:21,265::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:20:21,265::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:20:21,265::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:20:21,266::task::990::TaskManager.Task::(_decref) Task=`a82f1638-12a8-4405-8e83-d7a68232e107`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:20:21,758::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:20:22,854::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:22,934::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0367935 s, 27.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:20:24,777::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:20:24,946::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:25,021::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0308722 s, 33.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:20:27,045::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:27,120::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317276 s, 32.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:20:27,803::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:20:28,153::task::595::TaskManager.Task::(_updateState) Task=`4364a77d-01f2-4cda-b0b3-946af2499fe8`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:20:28,153::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:20:28,153::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00064667', 'lastCheck': '9.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:20:28,153::task::1185::TaskManager.Task::(prepare) Task=`4364a77d-01f2-4cda-b0b3-946af2499fe8`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00064667', 'lastCheck': '9.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:20:28,154::task::595::TaskManager.Task::(_updateState) Task=`4364a77d-01f2-4cda-b0b3-946af2499fe8`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:20:28,154::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:20:28,154::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:20:28,154::task::990::TaskManager.Task::(_decref) Task=`4364a77d-01f2-4cda-b0b3-946af2499fe8`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 12:20:28,609::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:20:28,616::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000546132 s, 7.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:20:29,133::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:29,207::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306632 s, 33.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:20:30,823::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:20:31,220::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 12:20:31,285::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:20:31,287::task::595::TaskManager.Task::(_updateState) Task=`4917f8f0-7f53-428b-b196-9723e7e07a19`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:20:31,288::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:20:31,290::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:20:31,291::task::1185::TaskManager.Task::(prepare) Task=`4917f8f0-7f53-428b-b196-9723e7e07a19`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:20:31,292::task::595::TaskManager.Task::(_updateState) Task=`4917f8f0-7f53-428b-b196-9723e7e07a19`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:20:31,292::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:20:31,293::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:20:31,294::task::990::TaskManager.Task::(_decref) Task=`4917f8f0-7f53-428b-b196-9723e7e07a19`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:20:31,301::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:20:31,303::task::595::TaskManager.Task::(_updateState) Task=`9cea31b0-def3-4e3a-b050-71711bce9fc9`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:20:31,304::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:20:31,306::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`05576c9c-8861-4e2b-b0fc-449fe14ba8d1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:20:31,307::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:20:31,308::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:20:31,309::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`05576c9c-8861-4e2b-b0fc-449fe14ba8d1`::Granted request >Thread-338::DEBUG::2014-05-13 12:20:31,311::task::827::TaskManager.Task::(resourceAcquired) Task=`9cea31b0-def3-4e3a-b050-71711bce9fc9`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:20:31,312::task::990::TaskManager.Task::(_decref) Task=`9cea31b0-def3-4e3a-b050-71711bce9fc9`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:20:31,316::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:20:31,316::task::1185::TaskManager.Task::(prepare) Task=`9cea31b0-def3-4e3a-b050-71711bce9fc9`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:20:31,317::task::595::TaskManager.Task::(_updateState) Task=`9cea31b0-def3-4e3a-b050-71711bce9fc9`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:20:31,318::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:20:31,319::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:20:31,320::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:20:31,322::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:20:31,322::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:20:31,323::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:20:31,324::task::990::TaskManager.Task::(_decref) Task=`9cea31b0-def3-4e3a-b050-71711bce9fc9`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:20:31,341::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0408786 s, 25.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:20:33,354::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:33,429::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305591 s, 33.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:20:33,841::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:20:35,442::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:35,856::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.370366 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:20:36,865::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:20:37,870::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:37,948::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362608 s, 28.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:20:38,623::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:20:38,631::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000769467 s, 5.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:20:39,884::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:20:39,961::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:40,041::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0366742 s, 27.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:20:41,350::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:20:41,350::task::595::TaskManager.Task::(_updateState) Task=`77db9040-d2e5-4287-9cbd-f59b47551245`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:20:41,351::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:20:41,351::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:20:41,351::task::1185::TaskManager.Task::(prepare) Task=`77db9040-d2e5-4287-9cbd-f59b47551245`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:20:41,351::task::595::TaskManager.Task::(_updateState) Task=`77db9040-d2e5-4287-9cbd-f59b47551245`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:20:41,351::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:20:41,352::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:20:41,352::task::990::TaskManager.Task::(_decref) Task=`77db9040-d2e5-4287-9cbd-f59b47551245`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:20:41,356::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:20:41,356::task::595::TaskManager.Task::(_updateState) Task=`e7c3f9e9-f76d-495c-9e2e-a1e480414a74`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:20:41,357::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:20:41,357::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`77b8260c-32c7-4991-969f-4cae55f48bdf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:20:41,357::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:20:41,357::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:20:41,357::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`77b8260c-32c7-4991-969f-4cae55f48bdf`::Granted request >Thread-338::DEBUG::2014-05-13 12:20:41,358::task::827::TaskManager.Task::(resourceAcquired) Task=`e7c3f9e9-f76d-495c-9e2e-a1e480414a74`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:20:41,358::task::990::TaskManager.Task::(_decref) Task=`e7c3f9e9-f76d-495c-9e2e-a1e480414a74`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:20:41,359::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:20:41,359::task::1185::TaskManager.Task::(prepare) Task=`e7c3f9e9-f76d-495c-9e2e-a1e480414a74`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:20:41,359::task::595::TaskManager.Task::(_updateState) Task=`e7c3f9e9-f76d-495c-9e2e-a1e480414a74`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:20:41,359::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:20:41,359::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:20:41,359::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:20:41,360::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:20:41,360::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:20:41,360::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:20:41,360::task::990::TaskManager.Task::(_decref) Task=`e7c3f9e9-f76d-495c-9e2e-a1e480414a74`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:20:42,054::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:42,128::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309428 s, 33.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:20:42,902::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:20:43,288::task::595::TaskManager.Task::(_updateState) Task=`3d5d4348-686d-4d3f-bdee-0771590c1f45`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:20:43,288::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:20:43,288::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000769467', 'lastCheck': '4.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:20:43,288::task::1185::TaskManager.Task::(prepare) Task=`3d5d4348-686d-4d3f-bdee-0771590c1f45`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000769467', 'lastCheck': '4.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:20:43,288::task::595::TaskManager.Task::(_updateState) Task=`3d5d4348-686d-4d3f-bdee-0771590c1f45`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:20:43,289::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:20:43,289::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:20:43,289::task::990::TaskManager.Task::(_decref) Task=`3d5d4348-686d-4d3f-bdee-0771590c1f45`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:20:44,141::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:44,214::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305434 s, 33.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:20:45,932::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:20:46,227::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:46,305::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358566 s, 28.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:20:48,318::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:48,405::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0441101 s, 23.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:20:48,639::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:20:48,647::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000663964 s, 6.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:20:48,950::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:20:50,418::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:50,491::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311355 s, 32.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:20:51,379::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [45b9a91f] >Thread-13::DEBUG::2014-05-13 12:20:51,379::task::595::TaskManager.Task::(_updateState) Task=`f3b52434-eeeb-4d9b-9f42-99cced9e1107`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:20:51,380::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:20:51,380::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:20:51,380::task::1185::TaskManager.Task::(prepare) Task=`f3b52434-eeeb-4d9b-9f42-99cced9e1107`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:20:51,380::task::595::TaskManager.Task::(_updateState) Task=`f3b52434-eeeb-4d9b-9f42-99cced9e1107`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:20:51,381::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:20:51,381::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:20:51,381::task::990::TaskManager.Task::(_decref) Task=`f3b52434-eeeb-4d9b-9f42-99cced9e1107`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:20:51,385::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [45b9a91f] >Thread-338::DEBUG::2014-05-13 12:20:51,386::task::595::TaskManager.Task::(_updateState) Task=`297452eb-2459-4a20-a34f-3541a8d0d939`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:20:51,386::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:20:51,386::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c3a05e97-ef1f-4939-a715-bce177977595`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:20:51,386::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:20:51,387::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:20:51,387::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c3a05e97-ef1f-4939-a715-bce177977595`::Granted request >Thread-338::DEBUG::2014-05-13 12:20:51,387::task::827::TaskManager.Task::(resourceAcquired) Task=`297452eb-2459-4a20-a34f-3541a8d0d939`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:20:51,387::task::990::TaskManager.Task::(_decref) Task=`297452eb-2459-4a20-a34f-3541a8d0d939`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:20:51,388::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:20:51,388::task::1185::TaskManager.Task::(prepare) Task=`297452eb-2459-4a20-a34f-3541a8d0d939`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:20:51,389::task::595::TaskManager.Task::(_updateState) Task=`297452eb-2459-4a20-a34f-3541a8d0d939`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:20:51,389::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:20:51,389::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:20:51,389::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:20:51,390::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:20:51,390::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:20:51,390::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:20:51,390::task::990::TaskManager.Task::(_decref) Task=`297452eb-2459-4a20-a34f-3541a8d0d939`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:20:51,971::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:20:52,504::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:52,582::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362934 s, 28.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:20:54,608::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:54,683::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309351 s, 33.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:20:54,998::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:20:56,696::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:56,770::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307079 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:20:58,016::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:20:58,432::task::595::TaskManager.Task::(_updateState) Task=`20507282-7953-4e83-bf70-f0ba93829b32`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:20:58,432::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:20:58,432::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000663964', 'lastCheck': '9.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:20:58,432::task::1185::TaskManager.Task::(prepare) Task=`20507282-7953-4e83-bf70-f0ba93829b32`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000663964', 'lastCheck': '9.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:20:58,433::task::595::TaskManager.Task::(_updateState) Task=`20507282-7953-4e83-bf70-f0ba93829b32`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:20:58,433::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:20:58,433::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:20:58,433::task::990::TaskManager.Task::(_decref) Task=`20507282-7953-4e83-bf70-f0ba93829b32`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 12:20:58,654::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:20:58,662::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000502407 s, 8.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:20:58,783::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:20:58,864::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383405 s, 26.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:21:00,877::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:21:00,959::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0385074 s, 26.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:21:01,033::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:21:01,411::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4478bd69] >Thread-13::DEBUG::2014-05-13 12:21:01,411::task::595::TaskManager.Task::(_updateState) Task=`24cec10c-c0d4-467c-a15c-4b1922f7b37a`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:21:01,411::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:21:01,412::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:21:01,412::task::1185::TaskManager.Task::(prepare) Task=`24cec10c-c0d4-467c-a15c-4b1922f7b37a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:21:01,412::task::595::TaskManager.Task::(_updateState) Task=`24cec10c-c0d4-467c-a15c-4b1922f7b37a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:21:01,412::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:21:01,412::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:21:01,413::task::990::TaskManager.Task::(_decref) Task=`24cec10c-c0d4-467c-a15c-4b1922f7b37a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:21:01,417::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4478bd69] >Thread-338::DEBUG::2014-05-13 12:21:01,418::task::595::TaskManager.Task::(_updateState) Task=`6abe359c-6095-463b-865b-30c1257d6fe2`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:21:01,418::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:21:01,418::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`bf982037-31bd-4565-bcef-e41e53333ccf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:21:01,418::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:21:01,419::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:21:01,419::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`bf982037-31bd-4565-bcef-e41e53333ccf`::Granted request >Thread-338::DEBUG::2014-05-13 12:21:01,419::task::827::TaskManager.Task::(resourceAcquired) Task=`6abe359c-6095-463b-865b-30c1257d6fe2`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:21:01,419::task::990::TaskManager.Task::(_decref) Task=`6abe359c-6095-463b-865b-30c1257d6fe2`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:21:01,420::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:21:01,420::task::1185::TaskManager.Task::(prepare) Task=`6abe359c-6095-463b-865b-30c1257d6fe2`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:21:01,421::task::595::TaskManager.Task::(_updateState) Task=`6abe359c-6095-463b-865b-30c1257d6fe2`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:21:01,421::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:21:01,421::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:21:01,421::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:21:01,421::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:21:01,421::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:21:01,422::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:21:01,422::task::990::TaskManager.Task::(_decref) Task=`6abe359c-6095-463b-865b-30c1257d6fe2`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:21:02,971::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:21:03,060::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0449129 s, 22.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:21:04,052::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:21:05,073::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:21:05,159::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315469 s, 32.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:21:07,075::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:21:07,173::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:21:07,252::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355317 s, 28.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:21:08,669::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:21:08,677::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000754316 s, 5.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:21:09,265::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:21:09,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359485 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:21:10,096::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:21:11,357::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:21:11,431::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311154 s, 32.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:21:11,441::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2ec76adc] >Thread-13::DEBUG::2014-05-13 12:21:11,443::task::595::TaskManager.Task::(_updateState) Task=`f687041e-d889-4403-be39-1bc79a0bbc88`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:21:11,444::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:21:11,445::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:21:11,446::task::1185::TaskManager.Task::(prepare) Task=`f687041e-d889-4403-be39-1bc79a0bbc88`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:21:11,446::task::595::TaskManager.Task::(_updateState) Task=`f687041e-d889-4403-be39-1bc79a0bbc88`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:21:11,447::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:21:11,448::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:21:11,449::task::990::TaskManager.Task::(_decref) Task=`f687041e-d889-4403-be39-1bc79a0bbc88`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:21:11,456::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2ec76adc] >Thread-338::DEBUG::2014-05-13 12:21:11,456::task::595::TaskManager.Task::(_updateState) Task=`9a126576-f620-4a75-bb4e-5e8f6cfad8f3`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:21:11,456::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:21:11,457::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f57a7aa7-533b-47f1-a424-4ffd10fe2101`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:21:11,457::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:21:11,457::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:21:11,457::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f57a7aa7-533b-47f1-a424-4ffd10fe2101`::Granted request >Thread-338::DEBUG::2014-05-13 12:21:11,458::task::827::TaskManager.Task::(resourceAcquired) Task=`9a126576-f620-4a75-bb4e-5e8f6cfad8f3`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:21:11,458::task::990::TaskManager.Task::(_decref) Task=`9a126576-f620-4a75-bb4e-5e8f6cfad8f3`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:21:11,459::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:21:11,459::task::1185::TaskManager.Task::(prepare) Task=`9a126576-f620-4a75-bb4e-5e8f6cfad8f3`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:21:11,459::task::595::TaskManager.Task::(_updateState) Task=`9a126576-f620-4a75-bb4e-5e8f6cfad8f3`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:21:11,459::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:21:11,459::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:21:11,459::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:21:11,460::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:21:11,460::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:21:11,460::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:21:11,460::task::990::TaskManager.Task::(_decref) Task=`9a126576-f620-4a75-bb4e-5e8f6cfad8f3`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:21:13,124::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:21:13,455::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:21:13,539::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0407683 s, 25.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:21:13,572::task::595::TaskManager.Task::(_updateState) Task=`53c7edc0-be43-4b01-9d38-b2618f715efa`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:21:13,572::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:21:13,572::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000754316', 'lastCheck': '4.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:21:13,573::task::1185::TaskManager.Task::(prepare) Task=`53c7edc0-be43-4b01-9d38-b2618f715efa`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000754316', 'lastCheck': '4.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:21:13,573::task::595::TaskManager.Task::(_updateState) Task=`53c7edc0-be43-4b01-9d38-b2618f715efa`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:21:13,573::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:21:13,573::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:21:13,573::task::990::TaskManager.Task::(_decref) Task=`53c7edc0-be43-4b01-9d38-b2618f715efa`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:21:15,552::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:21:15,626::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314389 s, 32.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:21:16,144::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:21:17,639::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:21:18,044::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.36166 s, 2.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:21:18,684::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:21:18,693::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000715306 s, 5.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:21:19,162::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:21:20,057::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:21:20,130::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312506 s, 32.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:21:21,481::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [303c64a0] >Thread-13::DEBUG::2014-05-13 12:21:21,482::task::595::TaskManager.Task::(_updateState) Task=`478103aa-674c-4c9d-812f-659bc8b7cf3c`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:21:21,482::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:21:21,482::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:21:21,482::task::1185::TaskManager.Task::(prepare) Task=`478103aa-674c-4c9d-812f-659bc8b7cf3c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:21:21,482::task::595::TaskManager.Task::(_updateState) Task=`478103aa-674c-4c9d-812f-659bc8b7cf3c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:21:21,483::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:21:21,483::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:21:21,483::task::990::TaskManager.Task::(_decref) Task=`478103aa-674c-4c9d-812f-659bc8b7cf3c`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:21:21,488::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [303c64a0] >Thread-338::DEBUG::2014-05-13 12:21:21,488::task::595::TaskManager.Task::(_updateState) Task=`b409ce0a-f89c-4c63-bf0f-2580f8b0e858`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:21:21,488::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:21:21,488::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e3175527-1ca4-4ea9-a42e-0ef4cb8d448d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:21:21,489::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:21:21,489::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:21:21,489::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e3175527-1ca4-4ea9-a42e-0ef4cb8d448d`::Granted request >Thread-338::DEBUG::2014-05-13 12:21:21,489::task::827::TaskManager.Task::(resourceAcquired) Task=`b409ce0a-f89c-4c63-bf0f-2580f8b0e858`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:21:21,490::task::990::TaskManager.Task::(_decref) Task=`b409ce0a-f89c-4c63-bf0f-2580f8b0e858`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:21:21,490::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:21:21,491::task::1185::TaskManager.Task::(prepare) Task=`b409ce0a-f89c-4c63-bf0f-2580f8b0e858`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:21:21,491::task::595::TaskManager.Task::(_updateState) Task=`b409ce0a-f89c-4c63-bf0f-2580f8b0e858`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:21:21,491::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:21:21,491::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:21:21,491::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:21:21,492::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:21:21,492::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:21:21,492::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:21:21,492::task::990::TaskManager.Task::(_decref) Task=`b409ce0a-f89c-4c63-bf0f-2580f8b0e858`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:21:22,143::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:21:22,180::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:21:22,224::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382947 s, 26.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:21:24,237::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:21:24,310::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307978 s, 33.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:21:25,206::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:21:26,323::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:21:26,397::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323967 s, 31.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:21:28,232::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:21:28,411::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:21:28,491::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0377278 s, 27.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:21:28,700::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:21:28,709::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00152351 s, 2.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:21:28,716::task::595::TaskManager.Task::(_updateState) Task=`66203aeb-2c25-42e8-909d-50c09313114a`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:21:28,717::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:21:28,718::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00152351', 'lastCheck': '0.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:21:28,718::task::1185::TaskManager.Task::(prepare) Task=`66203aeb-2c25-42e8-909d-50c09313114a`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00152351', 'lastCheck': '0.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:21:28,718::task::595::TaskManager.Task::(_updateState) Task=`66203aeb-2c25-42e8-909d-50c09313114a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:21:28,718::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:21:28,718::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:21:28,719::task::990::TaskManager.Task::(_decref) Task=`66203aeb-2c25-42e8-909d-50c09313114a`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:21:30,504::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:21:30,594::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0469311 s, 21.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:21:31,250::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:21:31,513::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:21:31,514::task::595::TaskManager.Task::(_updateState) Task=`64754798-37ac-470f-9437-d855518a5dc5`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:21:31,514::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:21:31,514::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:21:31,514::task::1185::TaskManager.Task::(prepare) Task=`64754798-37ac-470f-9437-d855518a5dc5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:21:31,515::task::595::TaskManager.Task::(_updateState) Task=`64754798-37ac-470f-9437-d855518a5dc5`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:21:31,515::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:21:31,515::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:21:31,515::task::990::TaskManager.Task::(_decref) Task=`64754798-37ac-470f-9437-d855518a5dc5`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:21:31,520::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:21:31,520::task::595::TaskManager.Task::(_updateState) Task=`4a707b78-8617-4c3c-8443-10bc28e8a2e7`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:21:31,521::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:21:31,521::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4d7ae873-435e-4d5a-bd25-3e51b2d21134`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:21:31,521::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:21:31,521::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:21:31,522::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4d7ae873-435e-4d5a-bd25-3e51b2d21134`::Granted request >Thread-338::DEBUG::2014-05-13 12:21:31,522::task::827::TaskManager.Task::(resourceAcquired) Task=`4a707b78-8617-4c3c-8443-10bc28e8a2e7`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:21:31,522::task::990::TaskManager.Task::(_decref) Task=`4a707b78-8617-4c3c-8443-10bc28e8a2e7`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:21:31,523::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:21:31,523::task::1185::TaskManager.Task::(prepare) Task=`4a707b78-8617-4c3c-8443-10bc28e8a2e7`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:21:31,523::task::595::TaskManager.Task::(_updateState) Task=`4a707b78-8617-4c3c-8443-10bc28e8a2e7`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:21:31,523::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:21:31,524::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:21:31,524::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:21:31,524::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:21:31,524::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:21:31,524::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:21:31,525::task::990::TaskManager.Task::(_decref) Task=`4a707b78-8617-4c3c-8443-10bc28e8a2e7`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:21:32,607::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:21:32,680::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314049 s, 32.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:21:34,274::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:21:34,693::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:21:34,774::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0372183 s, 27.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:21:36,787::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:21:36,865::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356781 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:21:37,301::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:21:38,720::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:21:38,728::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000613717 s, 6.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:21:38,879::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:21:38,960::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381176 s, 26.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:21:40,320::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:21:40,973::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:21:41,053::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365811 s, 28.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:21:41,545::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4765f2fb] >Thread-13::DEBUG::2014-05-13 12:21:41,546::task::595::TaskManager.Task::(_updateState) Task=`1737870b-222d-442d-929a-2ef682a91737`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:21:41,546::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:21:41,546::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:21:41,546::task::1185::TaskManager.Task::(prepare) Task=`1737870b-222d-442d-929a-2ef682a91737`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:21:41,547::task::595::TaskManager.Task::(_updateState) Task=`1737870b-222d-442d-929a-2ef682a91737`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:21:41,547::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:21:41,547::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:21:41,547::task::990::TaskManager.Task::(_decref) Task=`1737870b-222d-442d-929a-2ef682a91737`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:21:41,551::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4765f2fb] >Thread-338::DEBUG::2014-05-13 12:21:41,551::task::595::TaskManager.Task::(_updateState) Task=`f300fe2b-ab28-4c99-b884-6c958dc5ad62`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:21:41,552::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:21:41,553::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3fdc6570-4a21-441d-b2d0-8c2abe43e267`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:21:41,553::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:21:41,553::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:21:41,553::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3fdc6570-4a21-441d-b2d0-8c2abe43e267`::Granted request >Thread-338::DEBUG::2014-05-13 12:21:41,554::task::827::TaskManager.Task::(resourceAcquired) Task=`f300fe2b-ab28-4c99-b884-6c958dc5ad62`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:21:41,554::task::990::TaskManager.Task::(_decref) Task=`f300fe2b-ab28-4c99-b884-6c958dc5ad62`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:21:41,555::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:21:41,555::task::1185::TaskManager.Task::(prepare) Task=`f300fe2b-ab28-4c99-b884-6c958dc5ad62`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:21:41,555::task::595::TaskManager.Task::(_updateState) Task=`f300fe2b-ab28-4c99-b884-6c958dc5ad62`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:21:41,555::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:21:41,555::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:21:41,555::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:21:41,556::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:21:41,556::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:21:41,556::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:21:41,556::task::990::TaskManager.Task::(_decref) Task=`f300fe2b-ab28-4c99-b884-6c958dc5ad62`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:21:43,066::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:21:43,143::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320561 s, 31.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:21:43,343::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:21:43,854::task::595::TaskManager.Task::(_updateState) Task=`eb6b88f1-8ecc-46f9-8783-8cac2fc0564c`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:21:43,854::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:21:43,854::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000613717', 'lastCheck': '5.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:21:43,854::task::1185::TaskManager.Task::(prepare) Task=`eb6b88f1-8ecc-46f9-8783-8cac2fc0564c`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000613717', 'lastCheck': '5.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:21:43,855::task::595::TaskManager.Task::(_updateState) Task=`eb6b88f1-8ecc-46f9-8783-8cac2fc0564c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:21:43,855::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:21:43,855::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:21:43,855::task::990::TaskManager.Task::(_decref) Task=`eb6b88f1-8ecc-46f9-8783-8cac2fc0564c`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:21:45,157::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:21:45,245::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0447605 s, 22.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:21:46,363::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:21:47,258::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:21:47,338::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0349962 s, 29.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:21:48,738::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:21:48,749::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00257212 s, 1.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:21:49,351::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:21:49,382::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:21:49,442::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0413879 s, 24.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:21:51,458::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:21:51,540::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382839 s, 26.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:21:51,574::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:21:51,574::task::595::TaskManager.Task::(_updateState) Task=`4736d9b5-1933-43d1-bcb9-dce852d2df01`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:21:51,574::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:21:51,575::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:21:51,575::task::1185::TaskManager.Task::(prepare) Task=`4736d9b5-1933-43d1-bcb9-dce852d2df01`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:21:51,575::task::595::TaskManager.Task::(_updateState) Task=`4736d9b5-1933-43d1-bcb9-dce852d2df01`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:21:51,575::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:21:51,576::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:21:51,576::task::990::TaskManager.Task::(_decref) Task=`4736d9b5-1933-43d1-bcb9-dce852d2df01`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:21:51,580::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:21:51,581::task::595::TaskManager.Task::(_updateState) Task=`83b2dbbd-4a4d-4d86-b1cf-1e84a34381bb`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:21:51,581::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:21:51,581::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ed6872fa-d336-45cc-bb85-126c4eb007dc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:21:51,582::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:21:51,582::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:21:51,582::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ed6872fa-d336-45cc-bb85-126c4eb007dc`::Granted request >Thread-338::DEBUG::2014-05-13 12:21:51,582::task::827::TaskManager.Task::(resourceAcquired) Task=`83b2dbbd-4a4d-4d86-b1cf-1e84a34381bb`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:21:51,583::task::990::TaskManager.Task::(_decref) Task=`83b2dbbd-4a4d-4d86-b1cf-1e84a34381bb`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:21:51,583::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:21:51,584::task::1185::TaskManager.Task::(prepare) Task=`83b2dbbd-4a4d-4d86-b1cf-1e84a34381bb`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:21:51,584::task::595::TaskManager.Task::(_updateState) Task=`83b2dbbd-4a4d-4d86-b1cf-1e84a34381bb`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:21:51,584::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:21:51,584::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:21:51,584::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:21:51,585::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:21:51,585::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:21:51,585::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:21:51,585::task::990::TaskManager.Task::(_decref) Task=`83b2dbbd-4a4d-4d86-b1cf-1e84a34381bb`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:21:52,401::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:21:53,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:21:53,628::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310278 s, 33.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:21:55,420::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:21:55,641::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:21:55,720::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357032 s, 28.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:21:57,733::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:21:57,810::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0339811 s, 30.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:21:58,439::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:21:58,757::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:21:58,765::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.0006743 s, 6.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:21:58,989::task::595::TaskManager.Task::(_updateState) Task=`fc4cb82e-bcd5-4c4a-85fd-b2dceb031657`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:21:58,989::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:21:58,990::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0006743', 'lastCheck': '0.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:21:58,990::task::1185::TaskManager.Task::(prepare) Task=`fc4cb82e-bcd5-4c4a-85fd-b2dceb031657`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0006743', 'lastCheck': '0.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:21:58,990::task::595::TaskManager.Task::(_updateState) Task=`fc4cb82e-bcd5-4c4a-85fd-b2dceb031657`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:21:58,990::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:21:58,990::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:21:58,990::task::990::TaskManager.Task::(_decref) Task=`fc4cb82e-bcd5-4c4a-85fd-b2dceb031657`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:21:59,823::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:00,194::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.327613 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:22:01,457::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:22:01,603::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:22:01,603::task::595::TaskManager.Task::(_updateState) Task=`2de5a63f-6082-4e88-ac79-3d7c3c419472`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:22:01,603::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:22:01,604::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:22:01,604::task::1185::TaskManager.Task::(prepare) Task=`2de5a63f-6082-4e88-ac79-3d7c3c419472`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:22:01,604::task::595::TaskManager.Task::(_updateState) Task=`2de5a63f-6082-4e88-ac79-3d7c3c419472`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:22:01,604::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:22:01,604::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:22:01,605::task::990::TaskManager.Task::(_decref) Task=`2de5a63f-6082-4e88-ac79-3d7c3c419472`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:22:01,609::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:22:01,609::task::595::TaskManager.Task::(_updateState) Task=`dbb94a82-9b35-4c98-bd84-14309899f496`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:22:01,609::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:22:01,610::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a4a03ab2-acb2-42c3-8666-f7e29384877f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:22:01,610::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:22:01,610::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:22:01,610::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a4a03ab2-acb2-42c3-8666-f7e29384877f`::Granted request >Thread-338::DEBUG::2014-05-13 12:22:01,611::task::827::TaskManager.Task::(resourceAcquired) Task=`dbb94a82-9b35-4c98-bd84-14309899f496`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:22:01,611::task::990::TaskManager.Task::(_decref) Task=`dbb94a82-9b35-4c98-bd84-14309899f496`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:22:01,612::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:22:01,612::task::1185::TaskManager.Task::(prepare) Task=`dbb94a82-9b35-4c98-bd84-14309899f496`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:22:01,612::task::595::TaskManager.Task::(_updateState) Task=`dbb94a82-9b35-4c98-bd84-14309899f496`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:22:01,612::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:22:01,612::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:22:01,613::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:22:01,613::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:22:01,613::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:22:01,613::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:22:01,613::task::990::TaskManager.Task::(_decref) Task=`dbb94a82-9b35-4c98-bd84-14309899f496`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:22:02,206::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:02,281::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309765 s, 33.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:22:04,294::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:04,373::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352756 s, 29.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:22:04,490::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:22:06,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:06,467::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0372725 s, 27.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:22:07,511::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:22:08,480::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:08,554::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0308194 s, 33.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:22:08,773::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:22:08,780::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000687958 s, 6.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:22:10,530::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:22:10,567::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:10,938::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.328643 s, 3.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:22:11,631::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:22:11,631::task::595::TaskManager.Task::(_updateState) Task=`7ff71b26-b5fc-4ae3-953f-fdfb77dd06d2`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:22:11,631::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:22:11,632::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:22:11,632::task::1185::TaskManager.Task::(prepare) Task=`7ff71b26-b5fc-4ae3-953f-fdfb77dd06d2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:22:11,632::task::595::TaskManager.Task::(_updateState) Task=`7ff71b26-b5fc-4ae3-953f-fdfb77dd06d2`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:22:11,632::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:22:11,632::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:22:11,632::task::990::TaskManager.Task::(_decref) Task=`7ff71b26-b5fc-4ae3-953f-fdfb77dd06d2`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:22:11,637::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:22:11,637::task::595::TaskManager.Task::(_updateState) Task=`211c32dd-f712-4c0e-8862-21bd02c9a015`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:22:11,637::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:22:11,638::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4745476c-6013-4e24-b97f-3a6a2f255e34`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:22:11,638::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:22:11,638::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:22:11,638::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4745476c-6013-4e24-b97f-3a6a2f255e34`::Granted request >Thread-338::DEBUG::2014-05-13 12:22:11,639::task::827::TaskManager.Task::(resourceAcquired) Task=`211c32dd-f712-4c0e-8862-21bd02c9a015`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:22:11,639::task::990::TaskManager.Task::(_decref) Task=`211c32dd-f712-4c0e-8862-21bd02c9a015`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:22:11,640::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:22:11,640::task::1185::TaskManager.Task::(prepare) Task=`211c32dd-f712-4c0e-8862-21bd02c9a015`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:22:11,640::task::595::TaskManager.Task::(_updateState) Task=`211c32dd-f712-4c0e-8862-21bd02c9a015`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:22:11,640::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:22:11,640::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:22:11,640::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:22:11,641::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:22:11,641::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:22:11,641::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:22:11,641::task::990::TaskManager.Task::(_decref) Task=`211c32dd-f712-4c0e-8862-21bd02c9a015`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:22:12,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:13,026::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316191 s, 32.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:22:13,549::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:22:14,127::task::595::TaskManager.Task::(_updateState) Task=`d62f3e03-f0a9-48a7-b65f-c3b40d4d7739`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:22:14,127::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:22:14,127::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000687958', 'lastCheck': '5.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:22:14,128::task::1185::TaskManager.Task::(prepare) Task=`d62f3e03-f0a9-48a7-b65f-c3b40d4d7739`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000687958', 'lastCheck': '5.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:22:14,128::task::595::TaskManager.Task::(_updateState) Task=`d62f3e03-f0a9-48a7-b65f-c3b40d4d7739`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:22:14,128::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:22:14,128::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:22:14,128::task::990::TaskManager.Task::(_decref) Task=`d62f3e03-f0a9-48a7-b65f-c3b40d4d7739`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:22:15,040::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:15,113::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303592 s, 33.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:22:16,576::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:22:17,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:17,199::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307126 s, 33.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:22:18,787::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:22:18,795::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00060767 s, 6.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:22:19,216::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:19,295::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360357 s, 28.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:22:19,594::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:22:21,308::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:21,400::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.047453 s, 21.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:22:21,662::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:22:21,662::task::595::TaskManager.Task::(_updateState) Task=`a07e5973-e286-4af3-b48d-248e874f733e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:22:21,662::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:22:21,663::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:22:21,663::task::1185::TaskManager.Task::(prepare) Task=`a07e5973-e286-4af3-b48d-248e874f733e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:22:21,663::task::595::TaskManager.Task::(_updateState) Task=`a07e5973-e286-4af3-b48d-248e874f733e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:22:21,663::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:22:21,663::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:22:21,664::task::990::TaskManager.Task::(_decref) Task=`a07e5973-e286-4af3-b48d-248e874f733e`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:22:21,668::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:22:21,669::task::595::TaskManager.Task::(_updateState) Task=`b3d86048-9af0-4856-a4bd-e6bda51881dc`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:22:21,669::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:22:21,669::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9d1f008e-8739-41ce-bd87-b8322b732724`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:22:21,669::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:22:21,670::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:22:21,670::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9d1f008e-8739-41ce-bd87-b8322b732724`::Granted request >Thread-338::DEBUG::2014-05-13 12:22:21,670::task::827::TaskManager.Task::(resourceAcquired) Task=`b3d86048-9af0-4856-a4bd-e6bda51881dc`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:22:21,670::task::990::TaskManager.Task::(_decref) Task=`b3d86048-9af0-4856-a4bd-e6bda51881dc`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:22:21,671::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:22:21,671::task::1185::TaskManager.Task::(prepare) Task=`b3d86048-9af0-4856-a4bd-e6bda51881dc`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:22:21,671::task::595::TaskManager.Task::(_updateState) Task=`b3d86048-9af0-4856-a4bd-e6bda51881dc`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:22:21,672::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:22:21,672::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:22:21,672::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:22:21,672::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:22:21,672::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:22:21,673::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:22:21,673::task::990::TaskManager.Task::(_decref) Task=`b3d86048-9af0-4856-a4bd-e6bda51881dc`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:22:22,613::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:22:23,414::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:23,784::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.326856 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:22:25,633::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:22:25,797::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:25,876::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356716 s, 28.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:22:27,889::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:27,965::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0340281 s, 30.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:22:28,652::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:22:28,802::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:22:28,810::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000634335 s, 6.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:22:29,261::task::595::TaskManager.Task::(_updateState) Task=`77edd84a-696c-4b8d-bfc7-04f340656b88`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:22:29,261::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:22:29,261::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000634335', 'lastCheck': '0.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:22:29,261::task::1185::TaskManager.Task::(prepare) Task=`77edd84a-696c-4b8d-bfc7-04f340656b88`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000634335', 'lastCheck': '0.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:22:29,261::task::595::TaskManager.Task::(_updateState) Task=`77edd84a-696c-4b8d-bfc7-04f340656b88`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:22:29,261::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:22:29,262::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:22:29,262::task::990::TaskManager.Task::(_decref) Task=`77edd84a-696c-4b8d-bfc7-04f340656b88`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:22:29,978::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:30,052::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0332338 s, 30.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:22:31,669::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:22:31,691::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3179350e] >Thread-13::DEBUG::2014-05-13 12:22:31,692::task::595::TaskManager.Task::(_updateState) Task=`8b2f7d32-670e-4da4-ad2d-cb76de24774f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:22:31,692::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:22:31,692::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:22:31,692::task::1185::TaskManager.Task::(prepare) Task=`8b2f7d32-670e-4da4-ad2d-cb76de24774f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:22:31,692::task::595::TaskManager.Task::(_updateState) Task=`8b2f7d32-670e-4da4-ad2d-cb76de24774f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:22:31,693::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:22:31,693::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:22:31,693::task::990::TaskManager.Task::(_decref) Task=`8b2f7d32-670e-4da4-ad2d-cb76de24774f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:22:31,697::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3179350e] >Thread-338::DEBUG::2014-05-13 12:22:31,698::task::595::TaskManager.Task::(_updateState) Task=`7a9cf82f-4cc0-4024-96cb-effbb216eb28`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:22:31,698::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:22:31,698::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1b5510db-1a47-4ff5-a61b-b9e543a28e51`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:22:31,699::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:22:31,699::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:22:31,699::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1b5510db-1a47-4ff5-a61b-b9e543a28e51`::Granted request >Thread-338::DEBUG::2014-05-13 12:22:31,699::task::827::TaskManager.Task::(resourceAcquired) Task=`7a9cf82f-4cc0-4024-96cb-effbb216eb28`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:22:31,700::task::990::TaskManager.Task::(_decref) Task=`7a9cf82f-4cc0-4024-96cb-effbb216eb28`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:22:31,700::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:22:31,701::task::1185::TaskManager.Task::(prepare) Task=`7a9cf82f-4cc0-4024-96cb-effbb216eb28`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:22:31,701::task::595::TaskManager.Task::(_updateState) Task=`7a9cf82f-4cc0-4024-96cb-effbb216eb28`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:22:31,701::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:22:31,701::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:22:31,701::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:22:31,702::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:22:31,702::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:22:31,702::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:22:31,702::task::990::TaskManager.Task::(_decref) Task=`7a9cf82f-4cc0-4024-96cb-effbb216eb28`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:22:32,065::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:32,142::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384865 s, 26.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:22:34,150::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:34,522::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.329235 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:22:34,688::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:22:36,534::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:36,615::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0385481 s, 26.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:22:37,707::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:22:38,628::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:38,704::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0343618 s, 29.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:22:38,818::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:22:38,825::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000609332 s, 6.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:22:40,717::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:22:40,727::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:22:40,799::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0370324 s, 27.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:22:41,720::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:22:41,720::task::595::TaskManager.Task::(_updateState) Task=`9cd93c9f-1262-46df-adc4-1ebfb9750ca5`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:22:41,720::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:22:41,721::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:22:41,721::task::1185::TaskManager.Task::(prepare) Task=`9cd93c9f-1262-46df-adc4-1ebfb9750ca5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:22:41,721::task::595::TaskManager.Task::(_updateState) Task=`9cd93c9f-1262-46df-adc4-1ebfb9750ca5`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:22:41,721::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:22:41,721::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:22:41,721::task::990::TaskManager.Task::(_decref) Task=`9cd93c9f-1262-46df-adc4-1ebfb9750ca5`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:22:41,725::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:22:41,726::task::595::TaskManager.Task::(_updateState) Task=`7a8c4014-a450-47c2-b586-51eb60e55b3e`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:22:41,726::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:22:41,726::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a79ed6ca-a5fe-4929-8da3-a4cf3b38e9d6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:22:41,726::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:22:41,727::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:22:41,727::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a79ed6ca-a5fe-4929-8da3-a4cf3b38e9d6`::Granted request >Thread-338::DEBUG::2014-05-13 12:22:41,727::task::827::TaskManager.Task::(resourceAcquired) Task=`7a8c4014-a450-47c2-b586-51eb60e55b3e`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:22:41,727::task::990::TaskManager.Task::(_decref) Task=`7a8c4014-a450-47c2-b586-51eb60e55b3e`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:22:41,728::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:22:41,728::task::1185::TaskManager.Task::(prepare) Task=`7a8c4014-a450-47c2-b586-51eb60e55b3e`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:22:41,728::task::595::TaskManager.Task::(_updateState) Task=`7a8c4014-a450-47c2-b586-51eb60e55b3e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:22:41,728::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:22:41,729::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:22:41,729::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:22:41,729::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:22:41,729::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:22:41,729::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:22:41,729::task::990::TaskManager.Task::(_decref) Task=`7a8c4014-a450-47c2-b586-51eb60e55b3e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:22:42,813::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:42,887::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312512 s, 32.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:22:43,746::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:22:44,393::task::595::TaskManager.Task::(_updateState) Task=`257efd6d-918f-4f10-a5d7-b210498c138d`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:22:44,393::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:22:44,393::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000609332', 'lastCheck': '5.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:22:44,393::task::1185::TaskManager.Task::(prepare) Task=`257efd6d-918f-4f10-a5d7-b210498c138d`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000609332', 'lastCheck': '5.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:22:44,393::task::595::TaskManager.Task::(_updateState) Task=`257efd6d-918f-4f10-a5d7-b210498c138d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:22:44,393::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:22:44,394::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:22:44,394::task::990::TaskManager.Task::(_decref) Task=`257efd6d-918f-4f10-a5d7-b210498c138d`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:22:44,900::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:44,991::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.04837 s, 21.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:22:46,764::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:22:47,004::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:47,082::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035314 s, 29.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:22:48,832::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:22:48,840::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000609937 s, 6.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:22:49,095::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:49,173::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359181 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:22:49,783::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:22:51,186::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:51,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357849 s, 28.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:22:51,746::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [303c64a0] >Thread-13::DEBUG::2014-05-13 12:22:51,747::task::595::TaskManager.Task::(_updateState) Task=`ae49daac-8a92-4aa8-b10e-e4a58055f4eb`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:22:51,747::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:22:51,747::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:22:51,748::task::1185::TaskManager.Task::(prepare) Task=`ae49daac-8a92-4aa8-b10e-e4a58055f4eb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:22:51,748::task::595::TaskManager.Task::(_updateState) Task=`ae49daac-8a92-4aa8-b10e-e4a58055f4eb`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:22:51,748::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:22:51,748::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:22:51,748::task::990::TaskManager.Task::(_decref) Task=`ae49daac-8a92-4aa8-b10e-e4a58055f4eb`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:22:52,059::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [303c64a0] >Thread-338::DEBUG::2014-05-13 12:22:52,093::task::595::TaskManager.Task::(_updateState) Task=`f226b28c-c095-4b5a-a1d6-791722c3fd22`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:22:52,093::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:22:52,094::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a3081127-9ac7-4872-84e3-3951c2fe16b1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:22:52,094::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:22:52,094::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:22:52,094::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a3081127-9ac7-4872-84e3-3951c2fe16b1`::Granted request >Thread-338::DEBUG::2014-05-13 12:22:52,095::task::827::TaskManager.Task::(resourceAcquired) Task=`f226b28c-c095-4b5a-a1d6-791722c3fd22`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:22:52,096::task::990::TaskManager.Task::(_decref) Task=`f226b28c-c095-4b5a-a1d6-791722c3fd22`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:22:52,096::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:22:52,097::task::1185::TaskManager.Task::(prepare) Task=`f226b28c-c095-4b5a-a1d6-791722c3fd22`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:22:52,097::task::595::TaskManager.Task::(_updateState) Task=`f226b28c-c095-4b5a-a1d6-791722c3fd22`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:22:52,097::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:22:52,097::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:22:52,097::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:22:52,098::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:22:52,098::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:22:52,098::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:22:52,098::task::990::TaskManager.Task::(_decref) Task=`f226b28c-c095-4b5a-a1d6-791722c3fd22`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:22:52,806::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:22:53,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:53,352::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307002 s, 33.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:22:55,365::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:55,457::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362547 s, 28.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:22:55,822::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:22:57,471::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:57,546::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315251 s, 32.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:22:58,846::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:22:58,850::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:22:58,858::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000640238 s, 6.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:22:59,539::task::595::TaskManager.Task::(_updateState) Task=`ff8a28e6-4094-4c3a-9225-6a59f5677dce`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:22:59,539::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:22:59,539::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000640238', 'lastCheck': '0.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:22:59,540::task::1185::TaskManager.Task::(prepare) Task=`ff8a28e6-4094-4c3a-9225-6a59f5677dce`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000640238', 'lastCheck': '0.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:22:59,540::task::595::TaskManager.Task::(_updateState) Task=`ff8a28e6-4094-4c3a-9225-6a59f5677dce`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:22:59,540::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:22:59,540::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:22:59,540::task::990::TaskManager.Task::(_decref) Task=`ff8a28e6-4094-4c3a-9225-6a59f5677dce`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:22:59,559::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:22:59,919::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.318144 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:23:01,865::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:23:01,932::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:23:02,011::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0366632 s, 27.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:23:02,121::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4dbe430a] >Thread-13::DEBUG::2014-05-13 12:23:02,121::task::595::TaskManager.Task::(_updateState) Task=`c889a5d9-876e-4f55-ad40-41b869dab2b1`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:23:02,122::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:23:02,122::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:23:02,122::task::1185::TaskManager.Task::(prepare) Task=`c889a5d9-876e-4f55-ad40-41b869dab2b1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:23:02,122::task::595::TaskManager.Task::(_updateState) Task=`c889a5d9-876e-4f55-ad40-41b869dab2b1`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:23:02,122::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:23:02,123::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:23:02,123::task::990::TaskManager.Task::(_decref) Task=`c889a5d9-876e-4f55-ad40-41b869dab2b1`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:23:02,127::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4dbe430a] >Thread-338::DEBUG::2014-05-13 12:23:02,127::task::595::TaskManager.Task::(_updateState) Task=`85074711-1a5e-4f11-9f80-7e9b4c33bb46`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:23:02,128::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:23:02,128::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`70947ef5-024a-4762-8552-097a7cea841a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:23:02,128::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:23:02,128::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:23:02,129::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`70947ef5-024a-4762-8552-097a7cea841a`::Granted request >Thread-338::DEBUG::2014-05-13 12:23:02,129::task::827::TaskManager.Task::(resourceAcquired) Task=`85074711-1a5e-4f11-9f80-7e9b4c33bb46`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:23:02,129::task::990::TaskManager.Task::(_decref) Task=`85074711-1a5e-4f11-9f80-7e9b4c33bb46`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:23:02,130::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:23:02,130::task::1185::TaskManager.Task::(prepare) Task=`85074711-1a5e-4f11-9f80-7e9b4c33bb46`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:23:02,130::task::595::TaskManager.Task::(_updateState) Task=`85074711-1a5e-4f11-9f80-7e9b4c33bb46`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:23:02,130::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:23:02,131::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:23:02,131::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:23:02,131::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:23:02,131::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:23:02,131::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:23:02,132::task::990::TaskManager.Task::(_decref) Task=`85074711-1a5e-4f11-9f80-7e9b4c33bb46`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:23:04,024::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:23:04,098::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320409 s, 32.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:23:04,883::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:23:06,111::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:23:06,190::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317672 s, 32.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:23:07,902::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:23:08,205::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:23:08,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031262 s, 32.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:23:08,865::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:23:08,873::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000679005 s, 6.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:23:10,292::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:23:10,371::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035741 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:23:10,920::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:23:12,151::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3c71a9dd] >Thread-13::DEBUG::2014-05-13 12:23:12,151::task::595::TaskManager.Task::(_updateState) Task=`a795defd-95f1-440c-a074-997da921e642`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:23:12,152::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:23:12,152::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:23:12,152::task::1185::TaskManager.Task::(prepare) Task=`a795defd-95f1-440c-a074-997da921e642`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:23:12,152::task::595::TaskManager.Task::(_updateState) Task=`a795defd-95f1-440c-a074-997da921e642`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:23:12,152::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:23:12,153::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:23:12,153::task::990::TaskManager.Task::(_decref) Task=`a795defd-95f1-440c-a074-997da921e642`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:23:12,157::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3c71a9dd] >Thread-338::DEBUG::2014-05-13 12:23:12,158::task::595::TaskManager.Task::(_updateState) Task=`9c395f28-1abe-4c2a-8b5d-d2c5e4b245ca`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:23:12,158::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:23:12,158::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c74c8e17-9259-4e02-a6e7-56af814f81cf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:23:12,158::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:23:12,159::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:23:12,159::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c74c8e17-9259-4e02-a6e7-56af814f81cf`::Granted request >Thread-338::DEBUG::2014-05-13 12:23:12,159::task::827::TaskManager.Task::(resourceAcquired) Task=`9c395f28-1abe-4c2a-8b5d-d2c5e4b245ca`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:23:12,159::task::990::TaskManager.Task::(_decref) Task=`9c395f28-1abe-4c2a-8b5d-d2c5e4b245ca`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:23:12,160::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:23:12,160::task::1185::TaskManager.Task::(prepare) Task=`9c395f28-1abe-4c2a-8b5d-d2c5e4b245ca`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:23:12,161::task::595::TaskManager.Task::(_updateState) Task=`9c395f28-1abe-4c2a-8b5d-d2c5e4b245ca`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:23:12,161::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:23:12,161::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:23:12,161::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:23:12,161::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:23:12,162::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:23:12,162::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:23:12,162::task::990::TaskManager.Task::(_decref) Task=`9c395f28-1abe-4c2a-8b5d-d2c5e4b245ca`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:23:12,384::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:23:12,746::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.319058 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:23:13,939::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:23:14,671::task::595::TaskManager.Task::(_updateState) Task=`e6ff0692-1531-46e3-8371-d4996a89416f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:23:14,672::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:23:14,672::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000679005', 'lastCheck': '5.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:23:14,672::task::1185::TaskManager.Task::(prepare) Task=`e6ff0692-1531-46e3-8371-d4996a89416f`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000679005', 'lastCheck': '5.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:23:14,672::task::595::TaskManager.Task::(_updateState) Task=`e6ff0692-1531-46e3-8371-d4996a89416f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:23:14,672::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:23:14,672::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:23:14,672::task::990::TaskManager.Task::(_decref) Task=`e6ff0692-1531-46e3-8371-d4996a89416f`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:23:14,760::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:23:14,837::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0343391 s, 29.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:23:16,850::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:23:16,929::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357691 s, 28.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:23:16,959::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:23:18,879::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:23:18,887::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000605452 s, 6.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:23:18,942::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:23:19,023::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0388985 s, 26.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:23:19,978::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:23:21,036::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:23:21,110::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307373 s, 33.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:23:22,180::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6602059] >Thread-13::DEBUG::2014-05-13 12:23:22,212::task::595::TaskManager.Task::(_updateState) Task=`416c3af1-4550-48c9-add5-ed5b340f7186`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:23:22,212::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:23:22,213::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:23:22,213::task::1185::TaskManager.Task::(prepare) Task=`416c3af1-4550-48c9-add5-ed5b340f7186`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:23:22,213::task::595::TaskManager.Task::(_updateState) Task=`416c3af1-4550-48c9-add5-ed5b340f7186`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:23:22,213::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:23:22,213::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:23:22,213::task::990::TaskManager.Task::(_decref) Task=`416c3af1-4550-48c9-add5-ed5b340f7186`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:23:22,220::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6602059] >Thread-338::DEBUG::2014-05-13 12:23:22,221::task::595::TaskManager.Task::(_updateState) Task=`b5e326a5-20c7-4c8e-a722-eced23230af1`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:23:22,221::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:23:22,221::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a5f5f5a4-59b7-475d-bce3-3a695a4040e4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:23:22,221::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:23:22,222::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:23:22,222::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a5f5f5a4-59b7-475d-bce3-3a695a4040e4`::Granted request >Thread-338::DEBUG::2014-05-13 12:23:22,222::task::827::TaskManager.Task::(resourceAcquired) Task=`b5e326a5-20c7-4c8e-a722-eced23230af1`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:23:22,222::task::990::TaskManager.Task::(_decref) Task=`b5e326a5-20c7-4c8e-a722-eced23230af1`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:23:22,223::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:23:22,223::task::1185::TaskManager.Task::(prepare) Task=`b5e326a5-20c7-4c8e-a722-eced23230af1`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:23:22,224::task::595::TaskManager.Task::(_updateState) Task=`b5e326a5-20c7-4c8e-a722-eced23230af1`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:23:22,224::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:23:22,224::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:23:22,224::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:23:22,224::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:23:22,224::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:23:22,225::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:23:22,225::task::990::TaskManager.Task::(_decref) Task=`b5e326a5-20c7-4c8e-a722-eced23230af1`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:23:22,996::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:23:23,123::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:23:23,199::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0372587 s, 27.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:23:25,207::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:23:25,585::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.333778 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:23:26,014::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:23:27,598::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:23:27,680::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.038374 s, 26.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:23:28,895::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:23:28,902::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000593194 s, 6.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:23:29,033::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:23:29,693::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:23:29,767::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314159 s, 32.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:23:29,808::task::595::TaskManager.Task::(_updateState) Task=`d832313f-cacc-43be-9bd2-44b548860974`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:23:29,808::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:23:29,808::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000593194', 'lastCheck': '0.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:23:29,809::task::1185::TaskManager.Task::(prepare) Task=`d832313f-cacc-43be-9bd2-44b548860974`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000593194', 'lastCheck': '0.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:23:29,809::task::595::TaskManager.Task::(_updateState) Task=`d832313f-cacc-43be-9bd2-44b548860974`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:23:29,809::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:23:29,809::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:23:29,809::task::990::TaskManager.Task::(_decref) Task=`d832313f-cacc-43be-9bd2-44b548860974`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:23:31,780::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:23:31,860::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0378201 s, 27.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:23:32,051::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:23:32,244::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5fad7764] >Thread-13::DEBUG::2014-05-13 12:23:32,244::task::595::TaskManager.Task::(_updateState) Task=`e5762a98-bd95-4065-838d-54158ec26638`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:23:32,245::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:23:32,245::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:23:32,245::task::1185::TaskManager.Task::(prepare) Task=`e5762a98-bd95-4065-838d-54158ec26638`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:23:32,245::task::595::TaskManager.Task::(_updateState) Task=`e5762a98-bd95-4065-838d-54158ec26638`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:23:32,245::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:23:32,245::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:23:32,246::task::990::TaskManager.Task::(_decref) Task=`e5762a98-bd95-4065-838d-54158ec26638`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:23:32,253::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5fad7764] >Thread-338::DEBUG::2014-05-13 12:23:32,253::task::595::TaskManager.Task::(_updateState) Task=`e72c18b4-aa34-445f-a388-fa2395068c80`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:23:32,253::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:23:32,254::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c828dc9c-9026-44de-96f8-5f111f92c15d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:23:32,254::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:23:32,254::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:23:32,254::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c828dc9c-9026-44de-96f8-5f111f92c15d`::Granted request >Thread-338::DEBUG::2014-05-13 12:23:32,254::task::827::TaskManager.Task::(resourceAcquired) Task=`e72c18b4-aa34-445f-a388-fa2395068c80`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:23:32,255::task::990::TaskManager.Task::(_decref) Task=`e72c18b4-aa34-445f-a388-fa2395068c80`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:23:32,255::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:23:32,255::task::1185::TaskManager.Task::(prepare) Task=`e72c18b4-aa34-445f-a388-fa2395068c80`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:23:32,256::task::595::TaskManager.Task::(_updateState) Task=`e72c18b4-aa34-445f-a388-fa2395068c80`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:23:32,256::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:23:32,256::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:23:32,256::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:23:32,256::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:23:32,256::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:23:32,257::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:23:32,257::task::990::TaskManager.Task::(_decref) Task=`e72c18b4-aa34-445f-a388-fa2395068c80`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:23:33,873::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:23:33,948::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307085 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:23:35,069::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:23:35,961::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:23:36,036::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317688 s, 32.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:23:38,049::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:23:38,087::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:23:38,132::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0393932 s, 26.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:23:38,909::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:23:38,917::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000676099 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:23:40,145::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:23:40,225::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0377487 s, 27.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:23:41,106::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:23:42,238::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 12:23:42,276::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:23:42,278::task::595::TaskManager.Task::(_updateState) Task=`cbd7afb6-8e7c-418b-ae6d-9b2dd877c331`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:23:42,279::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:23:42,281::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:23:42,282::task::1185::TaskManager.Task::(prepare) Task=`cbd7afb6-8e7c-418b-ae6d-9b2dd877c331`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:23:42,283::task::595::TaskManager.Task::(_updateState) Task=`cbd7afb6-8e7c-418b-ae6d-9b2dd877c331`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:23:42,283::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:23:42,284::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:23:42,285::task::990::TaskManager.Task::(_decref) Task=`cbd7afb6-8e7c-418b-ae6d-9b2dd877c331`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:23:42,293::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:23:42,294::task::595::TaskManager.Task::(_updateState) Task=`7bd1ac9c-272b-451c-b15a-f629371e29ac`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:23:42,295::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:23:42,298::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d8777a07-0ddd-4d9b-a63f-a53e0dcd21ef`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:23:42,298::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:23:42,300::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:23:42,301::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d8777a07-0ddd-4d9b-a63f-a53e0dcd21ef`::Granted request >Thread-338::DEBUG::2014-05-13 12:23:42,303::task::827::TaskManager.Task::(resourceAcquired) Task=`7bd1ac9c-272b-451c-b15a-f629371e29ac`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:23:42,305::task::990::TaskManager.Task::(_decref) Task=`7bd1ac9c-272b-451c-b15a-f629371e29ac`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:23:42,311::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:23:42,313::task::1185::TaskManager.Task::(prepare) Task=`7bd1ac9c-272b-451c-b15a-f629371e29ac`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:23:42,314::task::595::TaskManager.Task::(_updateState) Task=`7bd1ac9c-272b-451c-b15a-f629371e29ac`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:23:42,316::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:23:42,317::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:23:42,320::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:23:42,322::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:23:42,323::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:23:42,324::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:23:42,326::task::990::TaskManager.Task::(_decref) Task=`7bd1ac9c-272b-451c-b15a-f629371e29ac`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:23:42,364::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0535512 s, 19.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:23:44,125::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:23:44,377::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:23:44,451::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315205 s, 32.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:23:44,967::task::595::TaskManager.Task::(_updateState) Task=`90f88bd9-cec6-426f-a405-976715f453e6`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:23:44,967::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:23:44,967::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000676099', 'lastCheck': '6.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:23:44,967::task::1185::TaskManager.Task::(prepare) Task=`90f88bd9-cec6-426f-a405-976715f453e6`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000676099', 'lastCheck': '6.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:23:44,968::task::595::TaskManager.Task::(_updateState) Task=`90f88bd9-cec6-426f-a405-976715f453e6`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:23:44,968::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:23:44,968::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:23:44,968::task::990::TaskManager.Task::(_decref) Task=`90f88bd9-cec6-426f-a405-976715f453e6`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:23:46,464::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:23:46,545::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0394557 s, 26.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:23:47,144::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:23:48,558::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:23:48,652::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0509438 s, 20.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:23:48,949::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:23:48,956::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000622363 s, 6.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:23:50,163::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:23:50,665::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:23:50,739::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311411 s, 32.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:23:52,352::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:23:52,353::task::595::TaskManager.Task::(_updateState) Task=`ab806f5f-e9c9-4602-af2e-d9700e26e618`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:23:52,353::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:23:52,353::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:23:52,353::task::1185::TaskManager.Task::(prepare) Task=`ab806f5f-e9c9-4602-af2e-d9700e26e618`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:23:52,354::task::595::TaskManager.Task::(_updateState) Task=`ab806f5f-e9c9-4602-af2e-d9700e26e618`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:23:52,354::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:23:52,354::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:23:52,354::task::990::TaskManager.Task::(_decref) Task=`ab806f5f-e9c9-4602-af2e-d9700e26e618`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:23:52,360::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:23:52,360::task::595::TaskManager.Task::(_updateState) Task=`96f9e3d4-2c48-4fe4-b6be-db548472d61a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:23:52,360::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:23:52,360::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a369e6ed-5d88-4c32-b30b-5e9c2378ead0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:23:52,361::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:23:52,361::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:23:52,361::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a369e6ed-5d88-4c32-b30b-5e9c2378ead0`::Granted request >Thread-338::DEBUG::2014-05-13 12:23:52,361::task::827::TaskManager.Task::(resourceAcquired) Task=`96f9e3d4-2c48-4fe4-b6be-db548472d61a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:23:52,362::task::990::TaskManager.Task::(_decref) Task=`96f9e3d4-2c48-4fe4-b6be-db548472d61a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:23:52,362::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:23:52,363::task::1185::TaskManager.Task::(prepare) Task=`96f9e3d4-2c48-4fe4-b6be-db548472d61a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:23:52,363::task::595::TaskManager.Task::(_updateState) Task=`96f9e3d4-2c48-4fe4-b6be-db548472d61a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:23:52,363::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:23:52,363::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:23:52,363::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:23:52,364::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:23:52,364::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:23:52,364::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:23:52,364::task::990::TaskManager.Task::(_decref) Task=`96f9e3d4-2c48-4fe4-b6be-db548472d61a`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:23:52,752::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:23:52,831::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356712 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:23:53,185::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:23:54,843::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:23:54,919::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319963 s, 32.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:23:56,206::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:23:56,933::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:23:57,013::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374477 s, 27.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:23:58,963::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:23:58,971::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000666814 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:23:59,026::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:23:59,106::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383732 s, 26.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:23:59,226::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:24:00,110::task::595::TaskManager.Task::(_updateState) Task=`2c455a9a-ed27-40b3-a7c5-438bbe842f8d`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:24:00,110::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:24:00,111::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000666814', 'lastCheck': '1.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:24:00,111::task::1185::TaskManager.Task::(prepare) Task=`2c455a9a-ed27-40b3-a7c5-438bbe842f8d`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000666814', 'lastCheck': '1.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:24:00,111::task::595::TaskManager.Task::(_updateState) Task=`2c455a9a-ed27-40b3-a7c5-438bbe842f8d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:24:00,111::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:24:00,111::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:24:00,111::task::990::TaskManager.Task::(_decref) Task=`2c455a9a-ed27-40b3-a7c5-438bbe842f8d`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:24:01,119::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:24:01,195::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0325839 s, 31.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:24:02,244::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:24:02,387::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:24:02,387::task::595::TaskManager.Task::(_updateState) Task=`eb297ce3-96a9-4f84-a6f3-bc4d3de98142`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:24:02,387::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:24:02,388::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:24:02,388::task::1185::TaskManager.Task::(prepare) Task=`eb297ce3-96a9-4f84-a6f3-bc4d3de98142`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:24:02,388::task::595::TaskManager.Task::(_updateState) Task=`eb297ce3-96a9-4f84-a6f3-bc4d3de98142`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:24:02,388::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:24:02,388::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:24:02,388::task::990::TaskManager.Task::(_decref) Task=`eb297ce3-96a9-4f84-a6f3-bc4d3de98142`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:24:02,393::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:24:02,394::task::595::TaskManager.Task::(_updateState) Task=`e76bf7a8-9541-4528-9a8b-6b34d4a241d4`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:24:02,394::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:24:02,394::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d84051c2-6e49-4109-949e-adb430987452`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:24:02,394::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:24:02,395::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:24:02,395::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d84051c2-6e49-4109-949e-adb430987452`::Granted request >Thread-338::DEBUG::2014-05-13 12:24:02,396::task::827::TaskManager.Task::(resourceAcquired) Task=`e76bf7a8-9541-4528-9a8b-6b34d4a241d4`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:24:02,396::task::990::TaskManager.Task::(_decref) Task=`e76bf7a8-9541-4528-9a8b-6b34d4a241d4`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:24:02,397::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:24:02,397::task::1185::TaskManager.Task::(prepare) Task=`e76bf7a8-9541-4528-9a8b-6b34d4a241d4`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:24:02,397::task::595::TaskManager.Task::(_updateState) Task=`e76bf7a8-9541-4528-9a8b-6b34d4a241d4`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:24:02,397::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:24:02,397::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:24:02,398::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:24:02,398::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:24:02,398::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:24:02,398::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:24:02,398::task::990::TaskManager.Task::(_decref) Task=`e76bf7a8-9541-4528-9a8b-6b34d4a241d4`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:24:03,207::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:24:03,297::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0486057 s, 21.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:24:05,272::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:24:05,310::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:24:05,385::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0332116 s, 30.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:24:07,411::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:24:07,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387231 s, 26.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:24:08,291::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:24:08,978::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:24:08,986::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000676973 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:24:09,509::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:24:09,591::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387384 s, 26.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:24:11,311::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:24:11,604::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:24:11,698::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0499106 s, 20.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:24:12,416::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:24:12,449::task::595::TaskManager.Task::(_updateState) Task=`6d56fcba-af8e-4ee2-beb6-610d7347b844`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:24:12,449::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:24:12,450::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:24:12,450::task::1185::TaskManager.Task::(prepare) Task=`6d56fcba-af8e-4ee2-beb6-610d7347b844`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:24:12,450::task::595::TaskManager.Task::(_updateState) Task=`6d56fcba-af8e-4ee2-beb6-610d7347b844`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:24:12,450::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:24:12,450::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:24:12,451::task::990::TaskManager.Task::(_decref) Task=`6d56fcba-af8e-4ee2-beb6-610d7347b844`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:24:12,456::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:24:12,457::task::595::TaskManager.Task::(_updateState) Task=`e7ed200e-ad3d-45b4-b32e-f1b71eb45141`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:24:12,457::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:24:12,457::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b5225812-b6ed-4533-8a17-c9e55bc5a3b0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:24:12,458::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:24:12,458::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:24:12,458::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b5225812-b6ed-4533-8a17-c9e55bc5a3b0`::Granted request >Thread-338::DEBUG::2014-05-13 12:24:12,458::task::827::TaskManager.Task::(resourceAcquired) Task=`e7ed200e-ad3d-45b4-b32e-f1b71eb45141`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:24:12,459::task::990::TaskManager.Task::(_decref) Task=`e7ed200e-ad3d-45b4-b32e-f1b71eb45141`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:24:12,459::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:24:12,459::task::1185::TaskManager.Task::(prepare) Task=`e7ed200e-ad3d-45b4-b32e-f1b71eb45141`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:24:12,460::task::595::TaskManager.Task::(_updateState) Task=`e7ed200e-ad3d-45b4-b32e-f1b71eb45141`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:24:12,460::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:24:12,460::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:24:12,460::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:24:12,460::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:24:12,461::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:24:12,461::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:24:12,461::task::990::TaskManager.Task::(_decref) Task=`e7ed200e-ad3d-45b4-b32e-f1b71eb45141`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:24:13,711::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:24:13,788::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0350096 s, 29.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:24:14,330::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:24:15,256::task::595::TaskManager.Task::(_updateState) Task=`cc86e1c8-83ab-4232-ad6d-1bb88a0b843c`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:24:15,256::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:24:15,257::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000676973', 'lastCheck': '6.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:24:15,257::task::1185::TaskManager.Task::(prepare) Task=`cc86e1c8-83ab-4232-ad6d-1bb88a0b843c`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000676973', 'lastCheck': '6.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:24:15,257::task::595::TaskManager.Task::(_updateState) Task=`cc86e1c8-83ab-4232-ad6d-1bb88a0b843c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:24:15,257::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:24:15,257::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:24:15,257::task::990::TaskManager.Task::(_decref) Task=`cc86e1c8-83ab-4232-ad6d-1bb88a0b843c`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:24:15,801::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:24:16,210::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.365942 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:24:17,349::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:24:18,223::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:24:18,299::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0336419 s, 30.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:24:18,993::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:24:19,001::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00058994 s, 6.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:24:20,313::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:24:20,374::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:24:20,390::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307718 s, 33.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:24:22,403::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:24:22,483::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0380215 s, 26.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:24:22,489::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [30b3ccf] >Thread-13::DEBUG::2014-05-13 12:24:22,490::task::595::TaskManager.Task::(_updateState) Task=`5ed6a6c8-96be-4dcb-9077-b7e28958e653`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:24:22,491::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:24:22,493::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:24:22,493::task::1185::TaskManager.Task::(prepare) Task=`5ed6a6c8-96be-4dcb-9077-b7e28958e653`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:24:22,494::task::595::TaskManager.Task::(_updateState) Task=`5ed6a6c8-96be-4dcb-9077-b7e28958e653`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:24:22,495::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:24:22,495::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:24:22,496::task::990::TaskManager.Task::(_decref) Task=`5ed6a6c8-96be-4dcb-9077-b7e28958e653`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:24:22,503::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [30b3ccf] >Thread-338::DEBUG::2014-05-13 12:24:22,504::task::595::TaskManager.Task::(_updateState) Task=`f1107443-852f-4a68-b94a-df8d6313bb79`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:24:22,505::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:24:22,507::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6b1c211c-9888-4ce4-9dcf-61c029a6f916`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:24:22,508::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:24:22,509::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:24:22,510::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6b1c211c-9888-4ce4-9dcf-61c029a6f916`::Granted request >Thread-338::DEBUG::2014-05-13 12:24:22,511::task::827::TaskManager.Task::(resourceAcquired) Task=`f1107443-852f-4a68-b94a-df8d6313bb79`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:24:22,512::task::990::TaskManager.Task::(_decref) Task=`f1107443-852f-4a68-b94a-df8d6313bb79`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:24:22,515::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:24:22,516::task::1185::TaskManager.Task::(prepare) Task=`f1107443-852f-4a68-b94a-df8d6313bb79`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:24:22,517::task::595::TaskManager.Task::(_updateState) Task=`f1107443-852f-4a68-b94a-df8d6313bb79`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:24:22,517::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:24:22,518::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:24:22,519::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:24:22,520::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:24:22,521::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:24:22,521::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:24:22,522::task::990::TaskManager.Task::(_decref) Task=`f1107443-852f-4a68-b94a-df8d6313bb79`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:24:23,395::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:24:24,530::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:24:24,605::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.033523 s, 30.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:24:26,415::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:24:26,619::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:24:26,697::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363327 s, 28.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:24:28,711::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:24:28,806::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.051825 s, 19.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:24:29,009::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:24:29,017::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00103221 s, 4.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:24:29,434::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:24:30,425::task::595::TaskManager.Task::(_updateState) Task=`0125dd7b-e50e-4e5e-a589-1600fc8579ca`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:24:30,425::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:24:30,425::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00103221', 'lastCheck': '1.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:24:30,426::task::1185::TaskManager.Task::(prepare) Task=`0125dd7b-e50e-4e5e-a589-1600fc8579ca`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00103221', 'lastCheck': '1.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:24:30,426::task::595::TaskManager.Task::(_updateState) Task=`0125dd7b-e50e-4e5e-a589-1600fc8579ca`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:24:30,426::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:24:30,426::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:24:30,426::task::990::TaskManager.Task::(_decref) Task=`0125dd7b-e50e-4e5e-a589-1600fc8579ca`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:24:30,818::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:24:30,893::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0327562 s, 31.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:24:32,452::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:24:32,543::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [dc1f215] >Thread-13::DEBUG::2014-05-13 12:24:32,543::task::595::TaskManager.Task::(_updateState) Task=`b0888483-fb0c-4cea-be69-64b6d6cdc258`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:24:32,543::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:24:32,544::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:24:32,544::task::1185::TaskManager.Task::(prepare) Task=`b0888483-fb0c-4cea-be69-64b6d6cdc258`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:24:32,544::task::595::TaskManager.Task::(_updateState) Task=`b0888483-fb0c-4cea-be69-64b6d6cdc258`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:24:32,544::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:24:32,544::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:24:32,544::task::990::TaskManager.Task::(_decref) Task=`b0888483-fb0c-4cea-be69-64b6d6cdc258`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:24:32,549::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [dc1f215] >Thread-338::DEBUG::2014-05-13 12:24:32,549::task::595::TaskManager.Task::(_updateState) Task=`2d580135-77f8-4a8c-84e6-5de8b5db6de1`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:24:32,549::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:24:32,549::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`eafea59c-8007-4c7d-931c-3d30f0ec627d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:24:32,550::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:24:32,550::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:24:32,550::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`eafea59c-8007-4c7d-931c-3d30f0ec627d`::Granted request >Thread-338::DEBUG::2014-05-13 12:24:32,550::task::827::TaskManager.Task::(resourceAcquired) Task=`2d580135-77f8-4a8c-84e6-5de8b5db6de1`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:24:32,550::task::990::TaskManager.Task::(_decref) Task=`2d580135-77f8-4a8c-84e6-5de8b5db6de1`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:24:32,551::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:24:32,551::task::1185::TaskManager.Task::(prepare) Task=`2d580135-77f8-4a8c-84e6-5de8b5db6de1`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:24:32,551::task::595::TaskManager.Task::(_updateState) Task=`2d580135-77f8-4a8c-84e6-5de8b5db6de1`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:24:32,552::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:24:32,552::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:24:32,552::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:24:32,552::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:24:32,552::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:24:32,552::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:24:32,553::task::990::TaskManager.Task::(_decref) Task=`2d580135-77f8-4a8c-84e6-5de8b5db6de1`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:24:32,906::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:24:32,982::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0338988 s, 30.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:24:34,995::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:24:35,075::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381301 s, 26.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:24:35,470::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:24:37,088::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:24:37,169::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0368538 s, 27.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:24:38,488::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:24:39,024::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:24:39,031::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000599923 s, 6.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:24:39,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:24:39,260::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365672 s, 28.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:24:41,274::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:24:41,349::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321679 s, 31.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:24:41,505::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:24:42,571::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:24:42,571::task::595::TaskManager.Task::(_updateState) Task=`07990d91-87f8-4dee-93d4-5419b5f5ab22`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:24:42,572::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:24:42,572::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:24:42,572::task::1185::TaskManager.Task::(prepare) Task=`07990d91-87f8-4dee-93d4-5419b5f5ab22`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:24:42,572::task::595::TaskManager.Task::(_updateState) Task=`07990d91-87f8-4dee-93d4-5419b5f5ab22`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:24:42,573::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:24:42,573::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:24:42,573::task::990::TaskManager.Task::(_decref) Task=`07990d91-87f8-4dee-93d4-5419b5f5ab22`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:24:42,618::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:24:42,618::task::595::TaskManager.Task::(_updateState) Task=`e3c6d902-4c9b-4ebd-9b58-5751515b7a8b`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:24:42,618::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:24:42,619::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`bcc4657e-1f41-49c6-b112-70f37f0e01b2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:24:42,619::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:24:42,619::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:24:42,619::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`bcc4657e-1f41-49c6-b112-70f37f0e01b2`::Granted request >Thread-338::DEBUG::2014-05-13 12:24:42,619::task::827::TaskManager.Task::(resourceAcquired) Task=`e3c6d902-4c9b-4ebd-9b58-5751515b7a8b`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:24:42,620::task::990::TaskManager.Task::(_decref) Task=`e3c6d902-4c9b-4ebd-9b58-5751515b7a8b`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:24:42,620::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:24:42,621::task::1185::TaskManager.Task::(prepare) Task=`e3c6d902-4c9b-4ebd-9b58-5751515b7a8b`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:24:42,621::task::595::TaskManager.Task::(_updateState) Task=`e3c6d902-4c9b-4ebd-9b58-5751515b7a8b`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:24:42,621::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:24:42,621::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:24:42,621::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:24:42,622::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:24:42,622::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:24:42,622::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:24:42,622::task::990::TaskManager.Task::(_decref) Task=`e3c6d902-4c9b-4ebd-9b58-5751515b7a8b`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:24:43,362::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:24:43,730::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.32382 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:24:44,524::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:24:45,552::task::595::TaskManager.Task::(_updateState) Task=`f0b55b56-c807-4aa6-aee5-b5ab0c86e7bb`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:24:45,552::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:24:45,553::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000599923', 'lastCheck': '6.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:24:45,553::task::1185::TaskManager.Task::(prepare) Task=`f0b55b56-c807-4aa6-aee5-b5ab0c86e7bb`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000599923', 'lastCheck': '6.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:24:45,553::task::595::TaskManager.Task::(_updateState) Task=`f0b55b56-c807-4aa6-aee5-b5ab0c86e7bb`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:24:45,553::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:24:45,553::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:24:45,553::task::990::TaskManager.Task::(_decref) Task=`f0b55b56-c807-4aa6-aee5-b5ab0c86e7bb`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:24:45,753::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:24:45,827::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309623 s, 33.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:24:47,542::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:24:47,840::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:24:47,916::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0331585 s, 30.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:24:49,039::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:24:49,047::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000801868 s, 5.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:24:49,929::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:24:50,009::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383175 s, 26.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:24:50,560::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:24:52,022::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:24:52,103::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0372656 s, 27.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:24:52,641::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6602059] >Thread-13::DEBUG::2014-05-13 12:24:52,642::task::595::TaskManager.Task::(_updateState) Task=`0a241352-a663-4f15-9338-01c639f2d35d`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:24:52,642::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:24:52,642::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:24:52,642::task::1185::TaskManager.Task::(prepare) Task=`0a241352-a663-4f15-9338-01c639f2d35d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:24:52,642::task::595::TaskManager.Task::(_updateState) Task=`0a241352-a663-4f15-9338-01c639f2d35d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:24:52,643::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:24:52,643::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:24:52,643::task::990::TaskManager.Task::(_decref) Task=`0a241352-a663-4f15-9338-01c639f2d35d`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:24:52,648::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6602059] >Thread-338::DEBUG::2014-05-13 12:24:52,648::task::595::TaskManager.Task::(_updateState) Task=`7358bb51-c470-4320-a80f-71f061ef3472`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:24:52,649::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:24:52,649::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9d81d4fb-7316-4aca-8b31-248ab24e0ee4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:24:52,649::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:24:52,649::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:24:52,650::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9d81d4fb-7316-4aca-8b31-248ab24e0ee4`::Granted request >Thread-338::DEBUG::2014-05-13 12:24:52,650::task::827::TaskManager.Task::(resourceAcquired) Task=`7358bb51-c470-4320-a80f-71f061ef3472`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:24:52,650::task::990::TaskManager.Task::(_decref) Task=`7358bb51-c470-4320-a80f-71f061ef3472`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:24:52,651::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:24:52,651::task::1185::TaskManager.Task::(prepare) Task=`7358bb51-c470-4320-a80f-71f061ef3472`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:24:52,651::task::595::TaskManager.Task::(_updateState) Task=`7358bb51-c470-4320-a80f-71f061ef3472`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:24:52,651::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:24:52,652::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:24:52,652::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:24:52,652::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:24:52,652::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:24:52,652::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:24:52,653::task::990::TaskManager.Task::(_decref) Task=`7358bb51-c470-4320-a80f-71f061ef3472`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:24:53,579::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:24:54,116::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:24:54,191::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0329518 s, 31.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:24:56,204::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:24:56,598::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:24:56,624::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.372998 s, 2.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:24:58,641::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:24:58,721::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359591 s, 28.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:24:59,054::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:24:59,062::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000733734 s, 5.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:24:59,616::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:25:00,680::task::595::TaskManager.Task::(_updateState) Task=`6f4caa36-94dc-4dfb-a3d5-a5d80bb58d2f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:25:00,680::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:25:00,681::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000733734', 'lastCheck': '1.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:25:00,681::task::1185::TaskManager.Task::(prepare) Task=`6f4caa36-94dc-4dfb-a3d5-a5d80bb58d2f`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000733734', 'lastCheck': '1.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:25:00,681::task::595::TaskManager.Task::(_updateState) Task=`6f4caa36-94dc-4dfb-a3d5-a5d80bb58d2f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:25:00,681::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:25:00,681::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:25:00,681::task::990::TaskManager.Task::(_decref) Task=`6f4caa36-94dc-4dfb-a3d5-a5d80bb58d2f`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:25:00,734::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:00,809::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316217 s, 32.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:25:02,635::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:25:02,669::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:25:02,670::task::595::TaskManager.Task::(_updateState) Task=`fe1a5b16-16cb-442a-ae41-6d6e55bbf5b6`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:25:02,670::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:25:02,670::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:25:02,671::task::1185::TaskManager.Task::(prepare) Task=`fe1a5b16-16cb-442a-ae41-6d6e55bbf5b6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:25:02,671::task::595::TaskManager.Task::(_updateState) Task=`fe1a5b16-16cb-442a-ae41-6d6e55bbf5b6`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:25:02,671::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:25:02,671::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:25:02,671::task::990::TaskManager.Task::(_decref) Task=`fe1a5b16-16cb-442a-ae41-6d6e55bbf5b6`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:25:02,676::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:25:02,676::task::595::TaskManager.Task::(_updateState) Task=`52759a95-48f8-4a42-b708-5e28a142f7d7`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:25:02,676::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:25:02,676::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`932533db-5522-4191-b42f-f8917fe62a27`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:25:02,677::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:25:02,677::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:25:02,677::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`932533db-5522-4191-b42f-f8917fe62a27`::Granted request >Thread-338::DEBUG::2014-05-13 12:25:02,677::task::827::TaskManager.Task::(resourceAcquired) Task=`52759a95-48f8-4a42-b708-5e28a142f7d7`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:25:02,677::task::990::TaskManager.Task::(_decref) Task=`52759a95-48f8-4a42-b708-5e28a142f7d7`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:25:02,678::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:25:02,678::task::1185::TaskManager.Task::(prepare) Task=`52759a95-48f8-4a42-b708-5e28a142f7d7`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:25:02,678::task::595::TaskManager.Task::(_updateState) Task=`52759a95-48f8-4a42-b708-5e28a142f7d7`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:25:02,679::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:25:02,679::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:25:02,679::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:25:02,679::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:25:02,679::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:25:02,679::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:25:02,680::task::990::TaskManager.Task::(_decref) Task=`52759a95-48f8-4a42-b708-5e28a142f7d7`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:25:02,821::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:02,896::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309983 s, 33.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:25:04,909::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:04,988::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357414 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:25:05,653::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:25:07,001::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:07,080::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036327 s, 28.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:25:08,672::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:25:09,066::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 41830dad-f012-4b4d-b636-863f66d5ca4b >Thread-322::DEBUG::2014-05-13 12:25:09,067::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-322::DEBUG::2014-05-13 12:25:09,069::blockSD::336::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:25:09,076::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000574083 s, 3.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:25:09,077::misc::287::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000574083 s, 3.6 MB/s'], size: 2048 >Thread-322::DEBUG::2014-05-13 12:25:09,077::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-322::WARNING::2014-05-13 12:25:09,077::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-322::DEBUG::2014-05-13 12:25:09,077::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-322::DEBUG::2014-05-13 12:25:09,078::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:25:09,078::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:25:09,078::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:25:09,079::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:25:09,079::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:25:09,080::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:09,093::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:09,193::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0557397 s, 18.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:25:09,329::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:25:09,330::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:25:09,331::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'LOCKPOLICY=', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', 'DESCRIPTION=ISCSI', 'LOGBLKSIZE=512', 'TYPE=ISCSI', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'PHYBLKSIZE=512', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', 'CLASS=Data', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'POOL_DESCRIPTION=DC_2', 'POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'MASTER_VERSION=1', 'ROLE=Master', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', '_SHA_CKSUM=28bb2d7cc60e0d779eb69fe93d050c16c769f069', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0'] >Thread-322::INFO::2014-05-13 12:25:09,332::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_imageNS already registered >Thread-322::INFO::2014-05-13 12:25:09,332::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_volumeNS already registered >Thread-322::INFO::2014-05-13 12:25:09,332::blockSD::452::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_lvmActivationNS already registered >Thread-322::DEBUG::2014-05-13 12:25:09,333::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 12:25:09,500::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:25:09,503::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:25:09,511::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000730133 s, 5.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:25:11,208::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:11,290::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0400742 s, 25.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:25:11,693::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:25:12,697::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:25:12,698::task::595::TaskManager.Task::(_updateState) Task=`67e79f0e-5274-4f07-840b-54f09db91c06`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:25:12,698::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:25:12,699::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:25:12,699::task::1185::TaskManager.Task::(prepare) Task=`67e79f0e-5274-4f07-840b-54f09db91c06`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:25:12,699::task::595::TaskManager.Task::(_updateState) Task=`67e79f0e-5274-4f07-840b-54f09db91c06`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:25:12,699::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:25:12,699::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:25:12,699::task::990::TaskManager.Task::(_decref) Task=`67e79f0e-5274-4f07-840b-54f09db91c06`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:25:12,704::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:25:12,705::task::595::TaskManager.Task::(_updateState) Task=`27bb229a-ecca-4c15-8a56-5942a19b9b9f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:25:12,705::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:25:12,705::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8404921a-45af-49dc-b330-f26abbe2d7f4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:25:12,705::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:25:12,706::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:25:12,706::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8404921a-45af-49dc-b330-f26abbe2d7f4`::Granted request >Thread-338::DEBUG::2014-05-13 12:25:12,706::task::827::TaskManager.Task::(resourceAcquired) Task=`27bb229a-ecca-4c15-8a56-5942a19b9b9f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:25:12,707::task::990::TaskManager.Task::(_decref) Task=`27bb229a-ecca-4c15-8a56-5942a19b9b9f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:25:12,707::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:25:12,707::task::1185::TaskManager.Task::(prepare) Task=`27bb229a-ecca-4c15-8a56-5942a19b9b9f`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:25:12,708::task::595::TaskManager.Task::(_updateState) Task=`27bb229a-ecca-4c15-8a56-5942a19b9b9f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:25:12,708::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:25:12,708::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:25:12,708::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:25:12,708::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:25:12,709::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:25:12,709::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:25:12,709::task::990::TaskManager.Task::(_decref) Task=`27bb229a-ecca-4c15-8a56-5942a19b9b9f`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:25:13,301::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:13,443::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0390383 s, 26.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:25:14,713::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:25:15,549::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:15,623::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311462 s, 32.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:25:15,815::task::595::TaskManager.Task::(_updateState) Task=`5333ba2b-a36d-4cab-83c7-f11b6f2bf0ff`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:25:15,816::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:25:15,818::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000730133', 'lastCheck': '6.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:25:15,819::task::1185::TaskManager.Task::(prepare) Task=`5333ba2b-a36d-4cab-83c7-f11b6f2bf0ff`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000730133', 'lastCheck': '6.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:25:15,820::task::595::TaskManager.Task::(_updateState) Task=`5333ba2b-a36d-4cab-83c7-f11b6f2bf0ff`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:25:15,821::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:25:15,822::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:25:15,823::task::990::TaskManager.Task::(_decref) Task=`5333ba2b-a36d-4cab-83c7-f11b6f2bf0ff`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:25:17,636::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:17,719::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0394728 s, 25.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:25:17,730::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:25:19,518::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:25:19,526::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000628512 s, 6.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:25:19,735::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:19,813::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035819 s, 28.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:25:20,747::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:25:21,826::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:21,911::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0423394 s, 24.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:25:22,727::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:25:22,728::task::595::TaskManager.Task::(_updateState) Task=`fc91b7f2-2104-4735-80a8-239b90530a3d`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:25:22,728::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:25:22,728::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:25:22,728::task::1185::TaskManager.Task::(prepare) Task=`fc91b7f2-2104-4735-80a8-239b90530a3d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:25:22,729::task::595::TaskManager.Task::(_updateState) Task=`fc91b7f2-2104-4735-80a8-239b90530a3d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:25:22,729::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:25:22,729::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:25:22,729::task::990::TaskManager.Task::(_decref) Task=`fc91b7f2-2104-4735-80a8-239b90530a3d`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:25:22,734::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:25:22,734::task::595::TaskManager.Task::(_updateState) Task=`4bf6bba5-d345-49e3-840b-53bfe6703b95`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:25:22,734::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:25:22,735::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`36f62058-41bd-4a4f-8635-b6cc289f0acd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:25:22,735::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:25:22,735::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:25:22,735::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`36f62058-41bd-4a4f-8635-b6cc289f0acd`::Granted request >Thread-338::DEBUG::2014-05-13 12:25:22,736::task::827::TaskManager.Task::(resourceAcquired) Task=`4bf6bba5-d345-49e3-840b-53bfe6703b95`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:25:22,736::task::990::TaskManager.Task::(_decref) Task=`4bf6bba5-d345-49e3-840b-53bfe6703b95`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:25:22,737::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:25:22,737::task::1185::TaskManager.Task::(prepare) Task=`4bf6bba5-d345-49e3-840b-53bfe6703b95`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:25:22,737::task::595::TaskManager.Task::(_updateState) Task=`4bf6bba5-d345-49e3-840b-53bfe6703b95`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:25:22,737::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:25:22,737::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:25:22,738::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:25:22,738::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:25:22,738::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:25:22,738::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:25:22,738::task::990::TaskManager.Task::(_decref) Task=`4bf6bba5-d345-49e3-840b-53bfe6703b95`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:25:23,767::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:25:23,924::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:24,009::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0429525 s, 23.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:25:26,022::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:26,102::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0371786 s, 27.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:25:26,786::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:25:28,116::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:28,196::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0377893 s, 27.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:25:29,533::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:25:29,545::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00572881 s, 715 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:25:29,808::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:25:30,209::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:30,291::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384994 s, 26.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:25:30,960::task::595::TaskManager.Task::(_updateState) Task=`c183501c-93de-4c14-acd8-afa65dda37eb`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:25:30,960::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:25:30,960::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00572881', 'lastCheck': '1.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:25:30,960::task::1185::TaskManager.Task::(prepare) Task=`c183501c-93de-4c14-acd8-afa65dda37eb`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00572881', 'lastCheck': '1.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:25:30,961::task::595::TaskManager.Task::(_updateState) Task=`c183501c-93de-4c14-acd8-afa65dda37eb`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:25:30,961::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:25:30,961::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:25:30,961::task::990::TaskManager.Task::(_decref) Task=`c183501c-93de-4c14-acd8-afa65dda37eb`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:25:32,304::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:32,380::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319298 s, 32.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:25:32,767::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:25:32,767::task::595::TaskManager.Task::(_updateState) Task=`e63c7bab-1e0b-433f-b686-0f2f500f9efc`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:25:32,767::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:25:32,768::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:25:32,768::task::1185::TaskManager.Task::(prepare) Task=`e63c7bab-1e0b-433f-b686-0f2f500f9efc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:25:32,768::task::595::TaskManager.Task::(_updateState) Task=`e63c7bab-1e0b-433f-b686-0f2f500f9efc`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:25:32,768::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:25:32,768::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:25:32,769::task::990::TaskManager.Task::(_decref) Task=`e63c7bab-1e0b-433f-b686-0f2f500f9efc`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:25:32,773::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:25:32,773::task::595::TaskManager.Task::(_updateState) Task=`e5b6c280-bfeb-4131-8e22-33d9243aedd7`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:25:32,773::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:25:32,774::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2a9afcde-3898-4108-99ba-7aedc03e1fcd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:25:32,774::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:25:32,774::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:25:32,775::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2a9afcde-3898-4108-99ba-7aedc03e1fcd`::Granted request >Thread-338::DEBUG::2014-05-13 12:25:32,775::task::827::TaskManager.Task::(resourceAcquired) Task=`e5b6c280-bfeb-4131-8e22-33d9243aedd7`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:25:32,775::task::990::TaskManager.Task::(_decref) Task=`e5b6c280-bfeb-4131-8e22-33d9243aedd7`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:25:32,776::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:25:32,776::task::1185::TaskManager.Task::(prepare) Task=`e5b6c280-bfeb-4131-8e22-33d9243aedd7`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:25:32,776::task::595::TaskManager.Task::(_updateState) Task=`e5b6c280-bfeb-4131-8e22-33d9243aedd7`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:25:32,776::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:25:32,777::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:25:32,777::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:25:32,777::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:25:32,777::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:25:32,777::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:25:32,778::task::990::TaskManager.Task::(_decref) Task=`e5b6c280-bfeb-4131-8e22-33d9243aedd7`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:25:32,825::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:25:34,393::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:34,474::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0385011 s, 26.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:25:35,842::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:25:36,487::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:36,904::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.375791 s, 2.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:25:38,863::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:25:38,917::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:38,992::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0334652 s, 30.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:25:39,552::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:25:39,560::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000657733 s, 6.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:25:41,006::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:41,086::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0366832 s, 27.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:25:41,882::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:25:42,797::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:25:42,798::task::595::TaskManager.Task::(_updateState) Task=`bd55901b-5d93-45e8-81fb-a546d6c6f888`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:25:42,798::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:25:42,798::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:25:42,798::task::1185::TaskManager.Task::(prepare) Task=`bd55901b-5d93-45e8-81fb-a546d6c6f888`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:25:42,799::task::595::TaskManager.Task::(_updateState) Task=`bd55901b-5d93-45e8-81fb-a546d6c6f888`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:25:42,799::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:25:42,799::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:25:42,799::task::990::TaskManager.Task::(_decref) Task=`bd55901b-5d93-45e8-81fb-a546d6c6f888`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:25:42,804::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:25:42,804::task::595::TaskManager.Task::(_updateState) Task=`e799547a-60b8-409b-a993-796fd1b229f2`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:25:42,804::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:25:42,805::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e4dc0c23-e3c0-4d26-ad08-bc5e759656e4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:25:42,805::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:25:42,805::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:25:42,806::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e4dc0c23-e3c0-4d26-ad08-bc5e759656e4`::Granted request >Thread-338::DEBUG::2014-05-13 12:25:42,806::task::827::TaskManager.Task::(resourceAcquired) Task=`e799547a-60b8-409b-a993-796fd1b229f2`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:25:42,806::task::990::TaskManager.Task::(_decref) Task=`e799547a-60b8-409b-a993-796fd1b229f2`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:25:42,807::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:25:42,807::task::1185::TaskManager.Task::(prepare) Task=`e799547a-60b8-409b-a993-796fd1b229f2`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:25:42,807::task::595::TaskManager.Task::(_updateState) Task=`e799547a-60b8-409b-a993-796fd1b229f2`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:25:42,807::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:25:42,808::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:25:42,808::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:25:42,808::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:25:42,808::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:25:42,808::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:25:42,809::task::990::TaskManager.Task::(_decref) Task=`e799547a-60b8-409b-a993-796fd1b229f2`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:25:43,099::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:43,179::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359579 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:25:44,900::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:25:45,192::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:45,265::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305588 s, 33.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:25:46,097::task::595::TaskManager.Task::(_updateState) Task=`a91a4791-7ec0-4854-b650-9bf532e16994`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:25:46,097::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:25:46,098::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000657733', 'lastCheck': '6.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:25:46,098::task::1185::TaskManager.Task::(prepare) Task=`a91a4791-7ec0-4854-b650-9bf532e16994`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000657733', 'lastCheck': '6.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:25:46,098::task::595::TaskManager.Task::(_updateState) Task=`a91a4791-7ec0-4854-b650-9bf532e16994`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:25:46,098::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:25:46,098::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:25:46,098::task::990::TaskManager.Task::(_decref) Task=`a91a4791-7ec0-4854-b650-9bf532e16994`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:25:47,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:47,354::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0388377 s, 26.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:25:47,922::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:25:49,362::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 12:25:49,568::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:25:49,576::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000860602 s, 4.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:25:49,760::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.355594 s, 2.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:25:50,939::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:25:51,773::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:51,849::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0342142 s, 29.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:25:52,827::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:25:52,827::task::595::TaskManager.Task::(_updateState) Task=`5526d68e-c1fc-4f64-bb35-636b6de6d2be`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:25:52,827::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:25:52,828::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:25:52,828::task::1185::TaskManager.Task::(prepare) Task=`5526d68e-c1fc-4f64-bb35-636b6de6d2be`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:25:52,828::task::595::TaskManager.Task::(_updateState) Task=`5526d68e-c1fc-4f64-bb35-636b6de6d2be`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:25:52,828::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:25:52,829::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:25:52,829::task::990::TaskManager.Task::(_decref) Task=`5526d68e-c1fc-4f64-bb35-636b6de6d2be`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:25:52,833::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:25:52,833::task::595::TaskManager.Task::(_updateState) Task=`90ae4008-273a-4c85-a89d-6ea74c5ba993`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:25:52,834::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:25:52,834::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c017412c-a9c9-44ea-a567-a4e8d8f1c52d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:25:52,834::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:25:52,834::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:25:52,835::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c017412c-a9c9-44ea-a567-a4e8d8f1c52d`::Granted request >Thread-338::DEBUG::2014-05-13 12:25:52,835::task::827::TaskManager.Task::(resourceAcquired) Task=`90ae4008-273a-4c85-a89d-6ea74c5ba993`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:25:52,835::task::990::TaskManager.Task::(_decref) Task=`90ae4008-273a-4c85-a89d-6ea74c5ba993`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:25:52,836::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:25:52,836::task::1185::TaskManager.Task::(prepare) Task=`90ae4008-273a-4c85-a89d-6ea74c5ba993`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:25:52,836::task::595::TaskManager.Task::(_updateState) Task=`90ae4008-273a-4c85-a89d-6ea74c5ba993`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:25:52,836::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:25:52,837::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:25:52,837::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:25:52,837::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:25:52,837::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:25:52,837::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:25:52,838::task::990::TaskManager.Task::(_decref) Task=`90ae4008-273a-4c85-a89d-6ea74c5ba993`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:25:53,861::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:53,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0338308 s, 30.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:25:53,957::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:25:55,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:56,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0386346 s, 26.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:25:56,986::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:25:58,061::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:25:58,142::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0388423 s, 26.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:25:59,582::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:25:59,590::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000795614 s, 5.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:26:00,005::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:26:00,155::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:00,235::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384029 s, 26.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:26:01,229::task::595::TaskManager.Task::(_updateState) Task=`a0666fe5-4b47-4ad7-842f-6b72e048cacc`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:26:01,268::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:26:01,268::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000795614', 'lastCheck': '1.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:26:01,269::task::1185::TaskManager.Task::(prepare) Task=`a0666fe5-4b47-4ad7-842f-6b72e048cacc`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000795614', 'lastCheck': '1.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:26:01,269::task::595::TaskManager.Task::(_updateState) Task=`a0666fe5-4b47-4ad7-842f-6b72e048cacc`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:26:01,269::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:26:01,269::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:26:01,269::task::990::TaskManager.Task::(_decref) Task=`a0666fe5-4b47-4ad7-842f-6b72e048cacc`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:26:02,248::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:02,329::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0391489 s, 26.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:26:02,855::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:26:02,855::task::595::TaskManager.Task::(_updateState) Task=`63f09d30-9ab8-4441-b51f-f80569eb7e47`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:26:02,856::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:26:02,856::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:26:02,856::task::1185::TaskManager.Task::(prepare) Task=`63f09d30-9ab8-4441-b51f-f80569eb7e47`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:26:02,856::task::595::TaskManager.Task::(_updateState) Task=`63f09d30-9ab8-4441-b51f-f80569eb7e47`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:26:02,856::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:26:02,857::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:26:02,857::task::990::TaskManager.Task::(_decref) Task=`63f09d30-9ab8-4441-b51f-f80569eb7e47`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:26:02,861::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:26:02,862::task::595::TaskManager.Task::(_updateState) Task=`fd031506-4431-43f6-8b85-7e24c5dc6961`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:26:02,862::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:26:02,862::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0115583c-fc94-49a4-b8c7-b0b0eb957b31`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:26:02,862::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:26:02,863::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:26:02,863::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0115583c-fc94-49a4-b8c7-b0b0eb957b31`::Granted request >Thread-338::DEBUG::2014-05-13 12:26:02,863::task::827::TaskManager.Task::(resourceAcquired) Task=`fd031506-4431-43f6-8b85-7e24c5dc6961`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:26:02,863::task::990::TaskManager.Task::(_decref) Task=`fd031506-4431-43f6-8b85-7e24c5dc6961`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:26:02,864::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:26:02,864::task::1185::TaskManager.Task::(prepare) Task=`fd031506-4431-43f6-8b85-7e24c5dc6961`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:26:02,864::task::595::TaskManager.Task::(_updateState) Task=`fd031506-4431-43f6-8b85-7e24c5dc6961`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:26:02,864::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:26:02,865::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:26:02,865::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:26:02,865::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:26:02,865::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:26:02,865::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:26:02,865::task::990::TaskManager.Task::(_decref) Task=`fd031506-4431-43f6-8b85-7e24c5dc6961`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:26:03,025::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:26:04,342::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:04,430::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0452001 s, 22.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:26:06,045::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:26:06,443::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:06,520::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0351252 s, 29.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:26:08,533::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:08,608::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0329454 s, 31.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:26:09,063::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:26:09,598::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:26:09,605::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000545287 s, 7.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:26:10,621::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:10,700::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0377312 s, 27.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:26:12,081::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:26:12,713::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:12,793::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360027 s, 28.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:26:12,884::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:26:12,884::task::595::TaskManager.Task::(_updateState) Task=`c4b01c7e-f33c-4c54-a9ee-89980f16c332`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:26:12,884::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:26:12,885::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:26:12,885::task::1185::TaskManager.Task::(prepare) Task=`c4b01c7e-f33c-4c54-a9ee-89980f16c332`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:26:12,885::task::595::TaskManager.Task::(_updateState) Task=`c4b01c7e-f33c-4c54-a9ee-89980f16c332`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:26:12,885::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:26:12,885::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:26:12,886::task::990::TaskManager.Task::(_decref) Task=`c4b01c7e-f33c-4c54-a9ee-89980f16c332`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:26:12,890::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:26:12,891::task::595::TaskManager.Task::(_updateState) Task=`d8ad2edd-871f-4e27-ad00-5ef9e7f160c8`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:26:12,891::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:26:12,891::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2eca3f20-ca77-4391-bd0f-977feac5c89d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:26:12,891::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:26:12,892::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:26:12,892::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2eca3f20-ca77-4391-bd0f-977feac5c89d`::Granted request >Thread-338::DEBUG::2014-05-13 12:26:12,892::task::827::TaskManager.Task::(resourceAcquired) Task=`d8ad2edd-871f-4e27-ad00-5ef9e7f160c8`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:26:12,892::task::990::TaskManager.Task::(_decref) Task=`d8ad2edd-871f-4e27-ad00-5ef9e7f160c8`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:26:12,893::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:26:12,893::task::1185::TaskManager.Task::(prepare) Task=`d8ad2edd-871f-4e27-ad00-5ef9e7f160c8`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:26:12,894::task::595::TaskManager.Task::(_updateState) Task=`d8ad2edd-871f-4e27-ad00-5ef9e7f160c8`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:26:12,894::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:26:12,894::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:26:12,894::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:26:12,894::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:26:12,894::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:26:12,895::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:26:12,895::task::990::TaskManager.Task::(_decref) Task=`d8ad2edd-871f-4e27-ad00-5ef9e7f160c8`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:26:14,806::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:14,879::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309473 s, 33.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:26:15,098::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:26:16,403::task::595::TaskManager.Task::(_updateState) Task=`4dbef1d2-be88-4ce2-b5c5-9b11c09fc9d9`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:26:16,404::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:26:16,404::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000545287', 'lastCheck': '6.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:26:16,404::task::1185::TaskManager.Task::(prepare) Task=`4dbef1d2-be88-4ce2-b5c5-9b11c09fc9d9`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000545287', 'lastCheck': '6.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:26:16,404::task::595::TaskManager.Task::(_updateState) Task=`4dbef1d2-be88-4ce2-b5c5-9b11c09fc9d9`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:26:16,404::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:26:16,404::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:26:16,405::task::990::TaskManager.Task::(_decref) Task=`4dbef1d2-be88-4ce2-b5c5-9b11c09fc9d9`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:26:16,892::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:17,300::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.364257 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:26:18,116::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:26:19,312::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:19,388::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0341802 s, 30.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:26:19,613::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:26:19,620::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000624361 s, 6.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:26:21,135::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:26:21,401::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:21,475::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310191 s, 33.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:26:22,914::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:26:22,914::task::595::TaskManager.Task::(_updateState) Task=`2756e5df-5896-49af-ac2a-4ff2685a2a79`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:26:22,914::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:26:22,915::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:26:22,915::task::1185::TaskManager.Task::(prepare) Task=`2756e5df-5896-49af-ac2a-4ff2685a2a79`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:26:22,915::task::595::TaskManager.Task::(_updateState) Task=`2756e5df-5896-49af-ac2a-4ff2685a2a79`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:26:22,915::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:26:22,915::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:26:22,916::task::990::TaskManager.Task::(_decref) Task=`2756e5df-5896-49af-ac2a-4ff2685a2a79`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:26:22,921::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:26:22,921::task::595::TaskManager.Task::(_updateState) Task=`7d7770d7-733e-4864-9316-ce0f19470366`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:26:22,921::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:26:22,922::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`88095d26-751c-47ef-9fa1-2160d6cbbba4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:26:22,922::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:26:22,922::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:26:22,922::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`88095d26-751c-47ef-9fa1-2160d6cbbba4`::Granted request >Thread-338::DEBUG::2014-05-13 12:26:22,922::task::827::TaskManager.Task::(resourceAcquired) Task=`7d7770d7-733e-4864-9316-ce0f19470366`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:26:22,923::task::990::TaskManager.Task::(_decref) Task=`7d7770d7-733e-4864-9316-ce0f19470366`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:26:22,923::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:26:22,924::task::1185::TaskManager.Task::(prepare) Task=`7d7770d7-733e-4864-9316-ce0f19470366`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:26:22,924::task::595::TaskManager.Task::(_updateState) Task=`7d7770d7-733e-4864-9316-ce0f19470366`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:26:22,924::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:26:22,924::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:26:22,924::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:26:22,925::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:26:22,925::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:26:22,925::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:26:22,925::task::990::TaskManager.Task::(_decref) Task=`7d7770d7-733e-4864-9316-ce0f19470366`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:26:23,488::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:23,563::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0336858 s, 30.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:26:24,154::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:26:25,576::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:25,656::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383067 s, 26.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:26:27,175::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:26:27,669::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:27,754::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387812 s, 26.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:26:29,627::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:26:29,635::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000671639 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:26:29,767::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:30,178::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.368424 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:26:30,193::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:26:31,535::task::595::TaskManager.Task::(_updateState) Task=`35b5074a-3ca8-4a3a-ba39-f9a25ad9b717`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:26:31,535::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:26:31,535::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000671639', 'lastCheck': '1.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:26:31,535::task::1185::TaskManager.Task::(prepare) Task=`35b5074a-3ca8-4a3a-ba39-f9a25ad9b717`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000671639', 'lastCheck': '1.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:26:31,536::task::595::TaskManager.Task::(_updateState) Task=`35b5074a-3ca8-4a3a-ba39-f9a25ad9b717`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:26:31,536::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:26:31,536::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:26:31,536::task::990::TaskManager.Task::(_decref) Task=`35b5074a-3ca8-4a3a-ba39-f9a25ad9b717`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:26:32,192::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:32,275::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0388404 s, 26.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:26:32,944::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5affc0d0] >Thread-13::DEBUG::2014-05-13 12:26:32,944::task::595::TaskManager.Task::(_updateState) Task=`42331d66-1549-43a3-a948-72dad6b98714`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:26:32,944::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:26:32,945::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:26:32,945::task::1185::TaskManager.Task::(prepare) Task=`42331d66-1549-43a3-a948-72dad6b98714`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:26:32,945::task::595::TaskManager.Task::(_updateState) Task=`42331d66-1549-43a3-a948-72dad6b98714`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:26:32,945::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:26:32,945::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:26:32,945::task::990::TaskManager.Task::(_decref) Task=`42331d66-1549-43a3-a948-72dad6b98714`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:26:32,950::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5affc0d0] >Thread-338::DEBUG::2014-05-13 12:26:32,950::task::595::TaskManager.Task::(_updateState) Task=`b5860f33-219b-450e-92b8-e0d5af567548`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:26:32,950::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:26:32,950::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2c8ef03f-7699-4fb2-85e1-3055e5955721`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:26:32,951::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:26:32,951::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:26:32,951::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2c8ef03f-7699-4fb2-85e1-3055e5955721`::Granted request >Thread-338::DEBUG::2014-05-13 12:26:32,951::task::827::TaskManager.Task::(resourceAcquired) Task=`b5860f33-219b-450e-92b8-e0d5af567548`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:26:32,951::task::990::TaskManager.Task::(_decref) Task=`b5860f33-219b-450e-92b8-e0d5af567548`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:26:32,952::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:26:32,952::task::1185::TaskManager.Task::(prepare) Task=`b5860f33-219b-450e-92b8-e0d5af567548`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:26:32,952::task::595::TaskManager.Task::(_updateState) Task=`b5860f33-219b-450e-92b8-e0d5af567548`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:26:32,952::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:26:32,953::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:26:32,953::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:26:32,953::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:26:32,953::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:26:32,953::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:26:32,953::task::990::TaskManager.Task::(_decref) Task=`b5860f33-219b-450e-92b8-e0d5af567548`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:26:33,209::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:26:34,288::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:34,363::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323167 s, 31.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:26:36,229::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:26:36,376::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:36,452::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320297 s, 32.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:26:38,465::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:38,538::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313622 s, 32.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:26:39,247::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:26:39,642::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:26:39,650::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000742939 s, 5.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:26:40,551::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:40,631::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381771 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:26:42,268::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:26:42,644::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:42,739::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0517556 s, 19.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:26:42,973::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:26:42,973::task::595::TaskManager.Task::(_updateState) Task=`b2177c32-d848-4624-a4b8-0a4c71a80703`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:26:42,974::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:26:42,974::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:26:42,974::task::1185::TaskManager.Task::(prepare) Task=`b2177c32-d848-4624-a4b8-0a4c71a80703`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:26:42,974::task::595::TaskManager.Task::(_updateState) Task=`b2177c32-d848-4624-a4b8-0a4c71a80703`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:26:42,974::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:26:42,975::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:26:42,975::task::990::TaskManager.Task::(_decref) Task=`b2177c32-d848-4624-a4b8-0a4c71a80703`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:26:42,980::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:26:42,980::task::595::TaskManager.Task::(_updateState) Task=`647fd762-e40d-432a-bfe3-3a0be15a6f8f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:26:42,980::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:26:42,981::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4f5bcf7b-afbc-46e3-acdc-88d3b625b2c9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:26:42,981::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:26:42,981::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:26:42,981::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4f5bcf7b-afbc-46e3-acdc-88d3b625b2c9`::Granted request >Thread-338::DEBUG::2014-05-13 12:26:42,982::task::827::TaskManager.Task::(resourceAcquired) Task=`647fd762-e40d-432a-bfe3-3a0be15a6f8f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:26:42,982::task::990::TaskManager.Task::(_decref) Task=`647fd762-e40d-432a-bfe3-3a0be15a6f8f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:26:42,983::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:26:42,983::task::1185::TaskManager.Task::(prepare) Task=`647fd762-e40d-432a-bfe3-3a0be15a6f8f`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:26:42,983::task::595::TaskManager.Task::(_updateState) Task=`647fd762-e40d-432a-bfe3-3a0be15a6f8f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:26:42,983::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:26:42,983::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:26:42,984::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:26:42,984::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:26:42,984::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:26:42,984::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:26:42,984::task::990::TaskManager.Task::(_decref) Task=`647fd762-e40d-432a-bfe3-3a0be15a6f8f`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:26:44,752::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:44,829::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0331449 s, 30.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:26:45,287::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:26:46,670::task::595::TaskManager.Task::(_updateState) Task=`61a5bcf9-2d6b-4265-bb0f-1b88e91b1c10`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:26:46,670::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:26:46,670::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000742939', 'lastCheck': '7.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:26:46,671::task::1185::TaskManager.Task::(prepare) Task=`61a5bcf9-2d6b-4265-bb0f-1b88e91b1c10`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000742939', 'lastCheck': '7.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:26:46,671::task::595::TaskManager.Task::(_updateState) Task=`61a5bcf9-2d6b-4265-bb0f-1b88e91b1c10`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:26:46,671::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:26:46,671::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:26:46,671::task::990::TaskManager.Task::(_decref) Task=`61a5bcf9-2d6b-4265-bb0f-1b88e91b1c10`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:26:46,842::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:46,923::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0394624 s, 25.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:26:48,306::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:26:48,937::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:49,011::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0330702 s, 31.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:26:49,658::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:26:49,666::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000720737 s, 5.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:26:51,024::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:51,101::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0344229 s, 29.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:26:51,324::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:26:53,009::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4765f2fb] >Thread-13::DEBUG::2014-05-13 12:26:53,010::task::595::TaskManager.Task::(_updateState) Task=`b1d793c8-e57f-4804-9cf0-005afe73a94c`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:26:53,010::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:26:53,010::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:26:53,010::task::1185::TaskManager.Task::(prepare) Task=`b1d793c8-e57f-4804-9cf0-005afe73a94c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:26:53,011::task::595::TaskManager.Task::(_updateState) Task=`b1d793c8-e57f-4804-9cf0-005afe73a94c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:26:53,011::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:26:53,011::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:26:53,011::task::990::TaskManager.Task::(_decref) Task=`b1d793c8-e57f-4804-9cf0-005afe73a94c`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:26:53,015::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4765f2fb] >Thread-338::DEBUG::2014-05-13 12:26:53,016::task::595::TaskManager.Task::(_updateState) Task=`4e052b9b-b61d-4470-8290-cb19089806dc`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:26:53,016::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:26:53,016::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ec44b419-9008-461b-b0f9-e4343bc4cd1a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:26:53,017::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:26:53,017::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:26:53,017::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ec44b419-9008-461b-b0f9-e4343bc4cd1a`::Granted request >Thread-338::DEBUG::2014-05-13 12:26:53,017::task::827::TaskManager.Task::(resourceAcquired) Task=`4e052b9b-b61d-4470-8290-cb19089806dc`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:26:53,018::task::990::TaskManager.Task::(_decref) Task=`4e052b9b-b61d-4470-8290-cb19089806dc`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:26:53,018::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:26:53,018::task::1185::TaskManager.Task::(prepare) Task=`4e052b9b-b61d-4470-8290-cb19089806dc`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:26:53,019::task::595::TaskManager.Task::(_updateState) Task=`4e052b9b-b61d-4470-8290-cb19089806dc`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:26:53,019::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:26:53,019::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:26:53,019::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:26:53,019::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:26:53,020::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:26:53,020::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:26:53,020::task::990::TaskManager.Task::(_decref) Task=`4e052b9b-b61d-4470-8290-cb19089806dc`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:26:53,114::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:53,195::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.039194 s, 26.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:26:54,345::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:26:55,208::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:55,296::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0468529 s, 21.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:26:57,310::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:26:57,366::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:26:57,395::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0386888 s, 26.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:26:59,408::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:26:59,482::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313863 s, 32.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:26:59,673::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:26:59,681::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000682799 s, 6.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:27:00,384::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:27:01,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:01,575::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0377043 s, 27.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:27:01,805::task::595::TaskManager.Task::(_updateState) Task=`c3f00b20-085a-43bd-9559-2626da12bc1f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:27:01,805::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:27:01,806::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000682799', 'lastCheck': '2.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:27:01,806::task::1185::TaskManager.Task::(prepare) Task=`c3f00b20-085a-43bd-9559-2626da12bc1f`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000682799', 'lastCheck': '2.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:27:01,806::task::595::TaskManager.Task::(_updateState) Task=`c3f00b20-085a-43bd-9559-2626da12bc1f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:27:01,806::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:27:01,806::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:27:01,807::task::990::TaskManager.Task::(_decref) Task=`c3f00b20-085a-43bd-9559-2626da12bc1f`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 12:27:03,043::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:27:03,044::task::595::TaskManager.Task::(_updateState) Task=`ee8c3ce1-934f-4567-842d-72467b930c04`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:27:03,044::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:27:03,044::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:27:03,044::task::1185::TaskManager.Task::(prepare) Task=`ee8c3ce1-934f-4567-842d-72467b930c04`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:27:03,044::task::595::TaskManager.Task::(_updateState) Task=`ee8c3ce1-934f-4567-842d-72467b930c04`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:27:03,045::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:27:03,045::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:27:03,045::task::990::TaskManager.Task::(_decref) Task=`ee8c3ce1-934f-4567-842d-72467b930c04`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:27:03,050::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:27:03,050::task::595::TaskManager.Task::(_updateState) Task=`6d86fb29-e8cd-430e-91b8-28110a4e996f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:27:03,051::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:27:03,051::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5d746136-d9c8-4e8e-ae39-3cba529f6ad2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:27:03,051::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:27:03,051::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:27:03,052::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5d746136-d9c8-4e8e-ae39-3cba529f6ad2`::Granted request >Thread-338::DEBUG::2014-05-13 12:27:03,052::task::827::TaskManager.Task::(resourceAcquired) Task=`6d86fb29-e8cd-430e-91b8-28110a4e996f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:27:03,052::task::990::TaskManager.Task::(_decref) Task=`6d86fb29-e8cd-430e-91b8-28110a4e996f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:27:03,053::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:27:03,053::task::1185::TaskManager.Task::(prepare) Task=`6d86fb29-e8cd-430e-91b8-28110a4e996f`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:27:03,053::task::595::TaskManager.Task::(_updateState) Task=`6d86fb29-e8cd-430e-91b8-28110a4e996f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:27:03,053::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:27:03,053::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:27:03,053::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:27:03,054::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:27:03,054::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:27:03,054::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:27:03,054::task::990::TaskManager.Task::(_decref) Task=`6d86fb29-e8cd-430e-91b8-28110a4e996f`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:27:03,402::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:27:03,588::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:03,662::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313254 s, 32.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:27:05,675::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:05,749::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.030722 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:27:06,421::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:27:07,762::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:07,842::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0368703 s, 27.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:27:09,439::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:27:09,688::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:27:09,695::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000747477 s, 5.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:27:09,855::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:10,153::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.243786 s, 4.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:27:12,166::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:12,250::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323553 s, 31.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:27:12,460::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:27:13,073::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:27:13,074::task::595::TaskManager.Task::(_updateState) Task=`e5a03e70-aee1-44c4-b0cd-79ae355777c2`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:27:13,074::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:27:13,074::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:27:13,074::task::1185::TaskManager.Task::(prepare) Task=`e5a03e70-aee1-44c4-b0cd-79ae355777c2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:27:13,075::task::595::TaskManager.Task::(_updateState) Task=`e5a03e70-aee1-44c4-b0cd-79ae355777c2`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:27:13,075::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:27:13,075::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:27:13,075::task::990::TaskManager.Task::(_decref) Task=`e5a03e70-aee1-44c4-b0cd-79ae355777c2`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:27:13,081::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:27:13,081::task::595::TaskManager.Task::(_updateState) Task=`0a7f9108-8c17-469e-af39-a1fc9f458ae7`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:27:13,081::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:27:13,082::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`09df2f71-2d66-4ccc-967b-c4e2d471a39b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:27:13,082::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:27:13,082::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:27:13,082::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`09df2f71-2d66-4ccc-967b-c4e2d471a39b`::Granted request >Thread-338::DEBUG::2014-05-13 12:27:13,083::task::827::TaskManager.Task::(resourceAcquired) Task=`0a7f9108-8c17-469e-af39-a1fc9f458ae7`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:27:13,083::task::990::TaskManager.Task::(_decref) Task=`0a7f9108-8c17-469e-af39-a1fc9f458ae7`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:27:13,084::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:27:13,084::task::1185::TaskManager.Task::(prepare) Task=`0a7f9108-8c17-469e-af39-a1fc9f458ae7`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:27:13,084::task::595::TaskManager.Task::(_updateState) Task=`0a7f9108-8c17-469e-af39-a1fc9f458ae7`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:27:13,084::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:27:13,084::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:27:13,085::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:27:13,085::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:27:13,085::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:27:13,085::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:27:13,085::task::990::TaskManager.Task::(_decref) Task=`0a7f9108-8c17-469e-af39-a1fc9f458ae7`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:27:14,264::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:14,342::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359055 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:27:15,477::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:27:16,355::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:16,431::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320219 s, 32.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:27:16,938::task::595::TaskManager.Task::(_updateState) Task=`710c8281-0491-4750-a024-b3204c521b88`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:27:16,938::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:27:16,939::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000747477', 'lastCheck': '7.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:27:16,939::task::1185::TaskManager.Task::(prepare) Task=`710c8281-0491-4750-a024-b3204c521b88`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000747477', 'lastCheck': '7.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:27:16,939::task::595::TaskManager.Task::(_updateState) Task=`710c8281-0491-4750-a024-b3204c521b88`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:27:16,939::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:27:16,939::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:27:16,939::task::990::TaskManager.Task::(_decref) Task=`710c8281-0491-4750-a024-b3204c521b88`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:27:18,444::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:27:18,496::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:27:18,521::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307732 s, 33.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:27:19,702::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:27:19,710::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000627677 s, 6.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:27:20,534::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:20,609::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0333714 s, 30.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:27:21,515::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:27:22,622::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:22,702::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379315 s, 27.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:27:23,105::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:27:23,105::task::595::TaskManager.Task::(_updateState) Task=`895ec143-5608-415e-8774-60eca60617c2`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:27:23,105::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:27:23,106::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:27:23,106::task::1185::TaskManager.Task::(prepare) Task=`895ec143-5608-415e-8774-60eca60617c2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:27:23,106::task::595::TaskManager.Task::(_updateState) Task=`895ec143-5608-415e-8774-60eca60617c2`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:27:23,106::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:27:23,106::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:27:23,107::task::990::TaskManager.Task::(_decref) Task=`895ec143-5608-415e-8774-60eca60617c2`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:27:23,111::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:27:23,111::task::595::TaskManager.Task::(_updateState) Task=`654a73f7-153a-4900-8411-2bea28896908`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:27:23,111::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:27:23,112::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`84dc77b3-cfc6-4024-a1ba-eb2319f3ad9a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:27:23,112::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:27:23,112::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:27:23,112::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`84dc77b3-cfc6-4024-a1ba-eb2319f3ad9a`::Granted request >Thread-338::DEBUG::2014-05-13 12:27:23,113::task::827::TaskManager.Task::(resourceAcquired) Task=`654a73f7-153a-4900-8411-2bea28896908`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:27:23,113::task::990::TaskManager.Task::(_decref) Task=`654a73f7-153a-4900-8411-2bea28896908`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:27:23,114::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:27:23,114::task::1185::TaskManager.Task::(prepare) Task=`654a73f7-153a-4900-8411-2bea28896908`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:27:23,114::task::595::TaskManager.Task::(_updateState) Task=`654a73f7-153a-4900-8411-2bea28896908`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:27:23,114::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:27:23,114::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:27:23,115::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:27:23,115::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:27:23,115::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:27:23,115::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:27:23,115::task::990::TaskManager.Task::(_decref) Task=`654a73f7-153a-4900-8411-2bea28896908`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:27:24,540::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:27:24,715::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:25,072::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.31363 s, 3.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:27:27,086::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:27,162::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318326 s, 32.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:27:27,565::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:27:29,175::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:29,255::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384536 s, 26.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:27:29,717::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:27:29,725::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000665217 s, 6.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:27:30,589::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:27:31,268::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:31,349::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0388539 s, 26.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:27:32,078::task::595::TaskManager.Task::(_updateState) Task=`ee45cd48-5319-419d-9018-7bcea044a233`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:27:32,079::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:27:32,079::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000665217', 'lastCheck': '2.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:27:32,079::task::1185::TaskManager.Task::(prepare) Task=`ee45cd48-5319-419d-9018-7bcea044a233`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000665217', 'lastCheck': '2.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:27:32,079::task::595::TaskManager.Task::(_updateState) Task=`ee45cd48-5319-419d-9018-7bcea044a233`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:27:32,079::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:27:32,079::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:27:32,080::task::990::TaskManager.Task::(_decref) Task=`ee45cd48-5319-419d-9018-7bcea044a233`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 12:27:33,133::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:27:33,133::task::595::TaskManager.Task::(_updateState) Task=`609da793-c6ba-4bb7-a621-776cf34b6c7f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:27:33,133::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:27:33,134::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:27:33,134::task::1185::TaskManager.Task::(prepare) Task=`609da793-c6ba-4bb7-a621-776cf34b6c7f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:27:33,134::task::595::TaskManager.Task::(_updateState) Task=`609da793-c6ba-4bb7-a621-776cf34b6c7f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:27:33,134::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:27:33,134::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:27:33,134::task::990::TaskManager.Task::(_decref) Task=`609da793-c6ba-4bb7-a621-776cf34b6c7f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:27:33,139::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:27:33,139::task::595::TaskManager.Task::(_updateState) Task=`5545bc19-98c9-4d46-9d1b-6538ad33914f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:27:33,139::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:27:33,140::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6a4d4603-26f8-478c-944a-eb533afd08c5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:27:33,140::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:27:33,140::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:27:33,140::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6a4d4603-26f8-478c-944a-eb533afd08c5`::Granted request >Thread-338::DEBUG::2014-05-13 12:27:33,141::task::827::TaskManager.Task::(resourceAcquired) Task=`5545bc19-98c9-4d46-9d1b-6538ad33914f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:27:33,141::task::990::TaskManager.Task::(_decref) Task=`5545bc19-98c9-4d46-9d1b-6538ad33914f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:27:33,141::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:27:33,141::task::1185::TaskManager.Task::(prepare) Task=`5545bc19-98c9-4d46-9d1b-6538ad33914f`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:27:33,142::task::595::TaskManager.Task::(_updateState) Task=`5545bc19-98c9-4d46-9d1b-6538ad33914f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:27:33,142::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:27:33,142::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:27:33,142::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:27:33,142::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:27:33,142::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:27:33,143::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:27:33,143::task::990::TaskManager.Task::(_decref) Task=`5545bc19-98c9-4d46-9d1b-6538ad33914f`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:27:33,362::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:33,437::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.032982 s, 31.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:27:33,607::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:27:35,450::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:35,525::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0331741 s, 30.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:27:36,628::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:27:37,538::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:37,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.313686 s, 3.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:27:39,652::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:27:39,732::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:27:39,739::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000640779 s, 6.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:27:39,907::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:39,987::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364922 s, 28.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:27:41,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:42,073::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310115 s, 33.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:27:42,671::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:27:43,162::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3256e43f] >Thread-13::DEBUG::2014-05-13 12:27:43,162::task::595::TaskManager.Task::(_updateState) Task=`c1d60aa3-a37e-4052-9eeb-765c4577327e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:27:43,162::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:27:43,163::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:27:43,163::task::1185::TaskManager.Task::(prepare) Task=`c1d60aa3-a37e-4052-9eeb-765c4577327e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:27:43,163::task::595::TaskManager.Task::(_updateState) Task=`c1d60aa3-a37e-4052-9eeb-765c4577327e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:27:43,163::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:27:43,163::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:27:43,164::task::990::TaskManager.Task::(_decref) Task=`c1d60aa3-a37e-4052-9eeb-765c4577327e`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:27:43,172::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3256e43f] >Thread-338::DEBUG::2014-05-13 12:27:43,172::task::595::TaskManager.Task::(_updateState) Task=`634ce6e7-fc76-49ec-a318-545712951957`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:27:43,172::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:27:43,173::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`7d37ea2a-1efe-4564-9527-d3dec0829464`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:27:43,173::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:27:43,173::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:27:43,173::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`7d37ea2a-1efe-4564-9527-d3dec0829464`::Granted request >Thread-338::DEBUG::2014-05-13 12:27:43,174::task::827::TaskManager.Task::(resourceAcquired) Task=`634ce6e7-fc76-49ec-a318-545712951957`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:27:43,174::task::990::TaskManager.Task::(_decref) Task=`634ce6e7-fc76-49ec-a318-545712951957`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:27:43,175::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:27:43,175::task::1185::TaskManager.Task::(prepare) Task=`634ce6e7-fc76-49ec-a318-545712951957`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:27:43,175::task::595::TaskManager.Task::(_updateState) Task=`634ce6e7-fc76-49ec-a318-545712951957`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:27:43,175::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:27:43,175::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:27:43,175::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:27:43,176::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:27:43,176::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:27:43,176::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:27:43,176::task::990::TaskManager.Task::(_decref) Task=`634ce6e7-fc76-49ec-a318-545712951957`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:27:44,087::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:44,165::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356347 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:27:45,690::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:27:46,179::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:46,263::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0304416 s, 33.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:27:47,209::task::595::TaskManager.Task::(_updateState) Task=`856c41fb-d5b3-4815-8309-3f2967874031`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:27:47,210::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:27:47,210::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000640779', 'lastCheck': '7.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:27:47,210::task::1185::TaskManager.Task::(prepare) Task=`856c41fb-d5b3-4815-8309-3f2967874031`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000640779', 'lastCheck': '7.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:27:47,210::task::595::TaskManager.Task::(_updateState) Task=`856c41fb-d5b3-4815-8309-3f2967874031`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:27:47,210::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:27:47,211::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:27:47,211::task::990::TaskManager.Task::(_decref) Task=`856c41fb-d5b3-4815-8309-3f2967874031`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:27:48,276::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:48,352::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322716 s, 31.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:27:48,708::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:27:49,746::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:27:49,754::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000680762 s, 6.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:27:50,365::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:50,445::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363445 s, 28.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:27:51,726::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:27:52,458::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:52,824::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.322677 s, 3.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:27:53,195::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:27:53,195::task::595::TaskManager.Task::(_updateState) Task=`8e378c88-f19a-4792-9fdb-2cb6fc62fde8`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:27:53,196::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:27:53,196::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:27:53,196::task::1185::TaskManager.Task::(prepare) Task=`8e378c88-f19a-4792-9fdb-2cb6fc62fde8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:27:53,196::task::595::TaskManager.Task::(_updateState) Task=`8e378c88-f19a-4792-9fdb-2cb6fc62fde8`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:27:53,197::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:27:53,197::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:27:53,197::task::990::TaskManager.Task::(_decref) Task=`8e378c88-f19a-4792-9fdb-2cb6fc62fde8`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:27:53,509::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:27:53,509::task::595::TaskManager.Task::(_updateState) Task=`afadc39a-925b-4e96-9fb0-ff562efa018d`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:27:53,509::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:27:53,510::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2f29d17b-0e16-469e-9685-99772072135a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:27:53,510::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:27:53,510::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:27:53,510::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2f29d17b-0e16-469e-9685-99772072135a`::Granted request >Thread-338::DEBUG::2014-05-13 12:27:53,510::task::827::TaskManager.Task::(resourceAcquired) Task=`afadc39a-925b-4e96-9fb0-ff562efa018d`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:27:53,511::task::990::TaskManager.Task::(_decref) Task=`afadc39a-925b-4e96-9fb0-ff562efa018d`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:27:53,511::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:27:53,511::task::1185::TaskManager.Task::(prepare) Task=`afadc39a-925b-4e96-9fb0-ff562efa018d`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:27:53,512::task::595::TaskManager.Task::(_updateState) Task=`afadc39a-925b-4e96-9fb0-ff562efa018d`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:27:53,512::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:27:53,512::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:27:53,512::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:27:53,512::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:27:53,512::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:27:53,513::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:27:53,513::task::990::TaskManager.Task::(_decref) Task=`afadc39a-925b-4e96-9fb0-ff562efa018d`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:27:54,744::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:27:54,837::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:54,911::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0308844 s, 33.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:27:56,924::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:56,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0334509 s, 30.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:27:57,764::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:27:59,012::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:27:59,092::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0380181 s, 26.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:27:59,761::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:27:59,769::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000645264 s, 6.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:28:00,785::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:28:01,105::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:28:01,186::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0388411 s, 26.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:28:02,544::task::595::TaskManager.Task::(_updateState) Task=`a602f278-3498-4f66-9737-1ca9ce025524`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:28:02,545::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:28:02,545::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000645264', 'lastCheck': '2.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:28:02,545::task::1185::TaskManager.Task::(prepare) Task=`a602f278-3498-4f66-9737-1ca9ce025524`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000645264', 'lastCheck': '2.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:28:02,545::task::595::TaskManager.Task::(_updateState) Task=`a602f278-3498-4f66-9737-1ca9ce025524`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:28:02,545::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:28:02,545::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:28:02,546::task::990::TaskManager.Task::(_decref) Task=`a602f278-3498-4f66-9737-1ca9ce025524`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:28:03,199::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 12:28:03,540::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:28:03,541::task::595::TaskManager.Task::(_updateState) Task=`15a309fd-4795-4591-9fdf-880a7b25fb17`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:28:03,541::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:28:03,541::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:28:03,542::task::1185::TaskManager.Task::(prepare) Task=`15a309fd-4795-4591-9fdf-880a7b25fb17`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:28:03,542::task::595::TaskManager.Task::(_updateState) Task=`15a309fd-4795-4591-9fdf-880a7b25fb17`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:28:03,542::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:28:03,542::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:28:03,542::task::990::TaskManager.Task::(_decref) Task=`15a309fd-4795-4591-9fdf-880a7b25fb17`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:28:03,547::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:28:03,548::task::595::TaskManager.Task::(_updateState) Task=`928f078f-f7d7-471f-b625-bba02b771ae4`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:28:03,548::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:28:03,548::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f693af4d-fc94-4b4d-b889-54c13dfdc7a9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:28:03,548::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:28:03,549::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:28:03,549::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f693af4d-fc94-4b4d-b889-54c13dfdc7a9`::Granted request >Thread-338::DEBUG::2014-05-13 12:28:03,551::task::827::TaskManager.Task::(resourceAcquired) Task=`928f078f-f7d7-471f-b625-bba02b771ae4`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:28:03,551::task::990::TaskManager.Task::(_decref) Task=`928f078f-f7d7-471f-b625-bba02b771ae4`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:28:03,552::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:28:03,552::task::1185::TaskManager.Task::(prepare) Task=`928f078f-f7d7-471f-b625-bba02b771ae4`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:28:03,553::task::595::TaskManager.Task::(_updateState) Task=`928f078f-f7d7-471f-b625-bba02b771ae4`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:28:03,553::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:28:03,553::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:28:03,553::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:28:03,554::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:28:03,554::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:28:03,554::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:28:03,555::task::990::TaskManager.Task::(_decref) Task=`928f078f-f7d7-471f-b625-bba02b771ae4`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:28:03,614::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.372077 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:28:03,802::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:28:05,627::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:28:05,701::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0325536 s, 31.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:28:06,820::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:28:07,714::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:28:07,794::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.03833 s, 26.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:28:09,776::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:28:09,784::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000743589 s, 5.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:28:09,807::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:28:09,839::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:28:09,890::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0419518 s, 24.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:28:11,903::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:28:11,978::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0326574 s, 31.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:28:12,858::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:28:13,573::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:28:13,574::task::595::TaskManager.Task::(_updateState) Task=`e60ed4ee-a66c-4964-b900-188acc63d9ce`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:28:13,574::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:28:13,574::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:28:13,575::task::1185::TaskManager.Task::(prepare) Task=`e60ed4ee-a66c-4964-b900-188acc63d9ce`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:28:13,575::task::595::TaskManager.Task::(_updateState) Task=`e60ed4ee-a66c-4964-b900-188acc63d9ce`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:28:13,575::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:28:13,575::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:28:13,575::task::990::TaskManager.Task::(_decref) Task=`e60ed4ee-a66c-4964-b900-188acc63d9ce`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:28:13,580::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:28:13,580::task::595::TaskManager.Task::(_updateState) Task=`273a6d70-c32c-407e-8849-7d1cc3653164`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:28:13,580::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:28:13,581::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5ea3383f-34d3-4a57-b660-bff12f5e01a1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:28:13,581::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:28:13,581::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:28:13,581::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5ea3383f-34d3-4a57-b660-bff12f5e01a1`::Granted request >Thread-338::DEBUG::2014-05-13 12:28:13,582::task::827::TaskManager.Task::(resourceAcquired) Task=`273a6d70-c32c-407e-8849-7d1cc3653164`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:28:13,582::task::990::TaskManager.Task::(_decref) Task=`273a6d70-c32c-407e-8849-7d1cc3653164`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:28:13,582::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:28:13,583::task::1185::TaskManager.Task::(prepare) Task=`273a6d70-c32c-407e-8849-7d1cc3653164`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:28:13,583::task::595::TaskManager.Task::(_updateState) Task=`273a6d70-c32c-407e-8849-7d1cc3653164`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:28:13,583::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:28:13,583::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:28:13,583::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:28:13,584::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:28:13,584::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:28:13,584::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:28:13,584::task::990::TaskManager.Task::(_decref) Task=`273a6d70-c32c-407e-8849-7d1cc3653164`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:28:13,991::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:28:14,389::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.355482 s, 2.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:28:15,877::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:28:16,402::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:28:16,475::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313098 s, 32.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:28:17,686::task::595::TaskManager.Task::(_updateState) Task=`443ef8d0-cb35-4cbd-9986-7d950989c1a7`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:28:17,686::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:28:17,687::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000743589', 'lastCheck': '7.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:28:17,687::task::1185::TaskManager.Task::(prepare) Task=`443ef8d0-cb35-4cbd-9986-7d950989c1a7`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000743589', 'lastCheck': '7.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:28:17,687::task::595::TaskManager.Task::(_updateState) Task=`443ef8d0-cb35-4cbd-9986-7d950989c1a7`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:28:17,687::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:28:17,687::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:28:17,687::task::990::TaskManager.Task::(_decref) Task=`443ef8d0-cb35-4cbd-9986-7d950989c1a7`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:28:18,489::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:28:18,568::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0366921 s, 27.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:28:18,897::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:28:19,791::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:28:19,799::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000734028 s, 5.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:28:20,581::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:28:20,656::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.032818 s, 31.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:28:21,917::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:28:22,669::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:28:22,750::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037999 s, 26.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:28:23,603::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:28:23,607::task::595::TaskManager.Task::(_updateState) Task=`2f24b378-e5c2-4113-8787-89632778e55c`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:28:23,608::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:28:23,608::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:28:23,608::task::1185::TaskManager.Task::(prepare) Task=`2f24b378-e5c2-4113-8787-89632778e55c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:28:23,608::task::595::TaskManager.Task::(_updateState) Task=`2f24b378-e5c2-4113-8787-89632778e55c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:28:23,608::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:28:23,609::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:28:23,609::task::990::TaskManager.Task::(_decref) Task=`2f24b378-e5c2-4113-8787-89632778e55c`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:28:23,614::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:28:23,615::task::595::TaskManager.Task::(_updateState) Task=`824d4554-7731-4f5a-8660-04a2b65692e8`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:28:23,615::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:28:23,615::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e084cde2-73ed-400e-9192-e754c992939e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:28:23,615::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:28:23,616::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:28:23,616::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e084cde2-73ed-400e-9192-e754c992939e`::Granted request >Thread-338::DEBUG::2014-05-13 12:28:23,616::task::827::TaskManager.Task::(resourceAcquired) Task=`824d4554-7731-4f5a-8660-04a2b65692e8`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:28:23,616::task::990::TaskManager.Task::(_decref) Task=`824d4554-7731-4f5a-8660-04a2b65692e8`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:28:23,617::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:28:23,617::task::1185::TaskManager.Task::(prepare) Task=`824d4554-7731-4f5a-8660-04a2b65692e8`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:28:23,617::task::595::TaskManager.Task::(_updateState) Task=`824d4554-7731-4f5a-8660-04a2b65692e8`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:28:23,617::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:28:23,618::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:28:23,618::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:28:23,618::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:28:23,618::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:28:23,618::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:28:23,618::task::990::TaskManager.Task::(_decref) Task=`824d4554-7731-4f5a-8660-04a2b65692e8`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:28:24,763::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:28:24,838::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316213 s, 32.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:28:24,935::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:28:26,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:28:26,926::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0340299 s, 30.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:28:27,955::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:28:28,939::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:28:29,310::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.329032 s, 3.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:28:29,806::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:28:29,814::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000673747 s, 6.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:28:30,972::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:28:31,323::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:28:31,403::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374199 s, 27.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:28:32,834::task::595::TaskManager.Task::(_updateState) Task=`1c585457-4515-4097-a7bc-cdd58864b557`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:28:32,835::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:28:32,835::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000673747', 'lastCheck': '3.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:28:32,835::task::1185::TaskManager.Task::(prepare) Task=`1c585457-4515-4097-a7bc-cdd58864b557`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000673747', 'lastCheck': '3.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:28:32,835::task::595::TaskManager.Task::(_updateState) Task=`1c585457-4515-4097-a7bc-cdd58864b557`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:28:32,836::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:28:32,836::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:28:32,836::task::990::TaskManager.Task::(_decref) Task=`1c585457-4515-4097-a7bc-cdd58864b557`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:28:33,416::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:28:33,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037524 s, 27.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:28:33,637::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:28:33,638::task::595::TaskManager.Task::(_updateState) Task=`1a06d911-f344-4198-be2a-5255e132e68f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:28:33,638::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:28:33,638::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:28:33,638::task::1185::TaskManager.Task::(prepare) Task=`1a06d911-f344-4198-be2a-5255e132e68f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:28:33,639::task::595::TaskManager.Task::(_updateState) Task=`1a06d911-f344-4198-be2a-5255e132e68f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:28:33,639::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:28:33,639::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:28:33,639::task::990::TaskManager.Task::(_decref) Task=`1a06d911-f344-4198-be2a-5255e132e68f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:28:33,645::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:28:33,645::task::595::TaskManager.Task::(_updateState) Task=`cd3774ee-d5f7-4f2d-a001-9ebe1a820429`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:28:33,645::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:28:33,646::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a965d9f4-78d1-409c-bdb1-97be4381b79b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:28:33,646::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:28:33,646::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:28:33,646::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a965d9f4-78d1-409c-bdb1-97be4381b79b`::Granted request >Thread-338::DEBUG::2014-05-13 12:28:33,647::task::827::TaskManager.Task::(resourceAcquired) Task=`cd3774ee-d5f7-4f2d-a001-9ebe1a820429`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:28:33,647::task::990::TaskManager.Task::(_decref) Task=`cd3774ee-d5f7-4f2d-a001-9ebe1a820429`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:28:33,648::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:28:33,648::task::1185::TaskManager.Task::(prepare) Task=`cd3774ee-d5f7-4f2d-a001-9ebe1a820429`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:28:33,648::task::595::TaskManager.Task::(_updateState) Task=`cd3774ee-d5f7-4f2d-a001-9ebe1a820429`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:28:33,648::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:28:33,648::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:28:33,649::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:28:33,649::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:28:33,649::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:28:33,649::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:28:33,649::task::990::TaskManager.Task::(_decref) Task=`cd3774ee-d5f7-4f2d-a001-9ebe1a820429`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:28:33,991::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:28:35,508::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:28:35,583::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0336281 s, 30.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:28:37,012::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:28:37,597::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:28:37,677::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0372531 s, 27.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:28:39,690::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 12:28:39,820::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:28:40,034::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:28:40,132::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.299663 s, 13.7 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:28:40,210::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.478917 s, 2.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:28:42,223::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:28:42,299::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321169 s, 31.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:28:43,055::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:28:43,667::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5fad7764] >Thread-13::DEBUG::2014-05-13 12:28:43,668::task::595::TaskManager.Task::(_updateState) Task=`af4ac459-95ff-4184-982a-f5d8fecd12ea`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:28:43,668::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:28:43,668::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:28:43,669::task::1185::TaskManager.Task::(prepare) Task=`af4ac459-95ff-4184-982a-f5d8fecd12ea`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:28:43,669::task::595::TaskManager.Task::(_updateState) Task=`af4ac459-95ff-4184-982a-f5d8fecd12ea`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:28:43,669::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:28:43,669::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:28:43,669::task::990::TaskManager.Task::(_decref) Task=`af4ac459-95ff-4184-982a-f5d8fecd12ea`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:28:43,674::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5fad7764] >Thread-338::DEBUG::2014-05-13 12:28:43,675::task::595::TaskManager.Task::(_updateState) Task=`efb3219a-c144-41cf-bf29-e8ed3431ab4d`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:28:43,675::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:28:43,675::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`19950f74-c289-4e78-baec-35aa30195f5d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:28:43,676::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:28:43,676::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:28:43,676::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`19950f74-c289-4e78-baec-35aa30195f5d`::Granted request >Thread-338::DEBUG::2014-05-13 12:28:43,676::task::827::TaskManager.Task::(resourceAcquired) Task=`efb3219a-c144-41cf-bf29-e8ed3431ab4d`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:28:43,676::task::990::TaskManager.Task::(_decref) Task=`efb3219a-c144-41cf-bf29-e8ed3431ab4d`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:28:43,677::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:28:43,677::task::1185::TaskManager.Task::(prepare) Task=`efb3219a-c144-41cf-bf29-e8ed3431ab4d`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:28:43,678::task::595::TaskManager.Task::(_updateState) Task=`efb3219a-c144-41cf-bf29-e8ed3431ab4d`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:28:43,678::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:28:43,678::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:28:43,678::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:28:43,678::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:28:43,679::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:28:43,679::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:28:43,679::task::990::TaskManager.Task::(_decref) Task=`efb3219a-c144-41cf-bf29-e8ed3431ab4d`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:28:44,312::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:28:44,404::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363808 s, 28.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:28:46,073::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:28:46,417::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:28:46,493::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0334082 s, 30.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:28:47,970::task::595::TaskManager.Task::(_updateState) Task=`af68648a-f566-4c66-a293-a0aba09f931b`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:28:47,970::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:28:47,970::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.299663', 'lastCheck': '7.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:28:47,970::task::1185::TaskManager.Task::(prepare) Task=`af68648a-f566-4c66-a293-a0aba09f931b`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.299663', 'lastCheck': '7.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:28:47,970::task::595::TaskManager.Task::(_updateState) Task=`af68648a-f566-4c66-a293-a0aba09f931b`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:28:47,971::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:28:47,971::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:28:47,971::task::990::TaskManager.Task::(_decref) Task=`af68648a-f566-4c66-a293-a0aba09f931b`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:28:48,506::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:28:48,587::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.038387 s, 26.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:28:49,091::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:28:50,153::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:28:50,162::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000817143 s, 5.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:28:50,600::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:28:50,675::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0340062 s, 30.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:28:52,111::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:28:52,688::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:28:52,771::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0366038 s, 28.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:28:53,697::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:28:53,697::task::595::TaskManager.Task::(_updateState) Task=`9544ff20-53c4-4530-bef5-20edc6d84e44`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:28:53,697::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:28:53,698::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:28:53,698::task::1185::TaskManager.Task::(prepare) Task=`9544ff20-53c4-4530-bef5-20edc6d84e44`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:28:53,698::task::595::TaskManager.Task::(_updateState) Task=`9544ff20-53c4-4530-bef5-20edc6d84e44`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:28:53,698::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:28:53,698::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:28:53,699::task::990::TaskManager.Task::(_decref) Task=`9544ff20-53c4-4530-bef5-20edc6d84e44`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:28:53,703::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:28:53,703::task::595::TaskManager.Task::(_updateState) Task=`d6b4158e-547b-480f-8936-d185f756b3c7`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:28:53,704::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:28:53,704::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`372c5b3d-45f7-4c66-8767-2ec62fc54b6c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:28:53,704::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:28:53,704::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:28:53,704::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`372c5b3d-45f7-4c66-8767-2ec62fc54b6c`::Granted request >Thread-338::DEBUG::2014-05-13 12:28:53,705::task::827::TaskManager.Task::(resourceAcquired) Task=`d6b4158e-547b-480f-8936-d185f756b3c7`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:28:53,705::task::990::TaskManager.Task::(_decref) Task=`d6b4158e-547b-480f-8936-d185f756b3c7`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:28:53,706::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:28:53,706::task::1185::TaskManager.Task::(prepare) Task=`d6b4158e-547b-480f-8936-d185f756b3c7`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:28:53,706::task::595::TaskManager.Task::(_updateState) Task=`d6b4158e-547b-480f-8936-d185f756b3c7`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:28:53,706::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:28:53,706::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:28:53,706::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:28:53,707::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:28:53,707::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:28:53,707::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:28:53,707::task::990::TaskManager.Task::(_decref) Task=`d6b4158e-547b-480f-8936-d185f756b3c7`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:28:54,784::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:28:54,869::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0429113 s, 23.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:28:55,131::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:28:56,881::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:28:56,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0336607 s, 30.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:28:58,150::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:28:58,977::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:28:59,058::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381915 s, 26.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:29:00,169::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:29:00,176::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000714586 s, 5.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:29:01,072::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:29:01,168::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:29:01,474::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.359566 s, 2.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:29:03,110::task::595::TaskManager.Task::(_updateState) Task=`7b58c4ab-d546-485b-8144-0ba5bd02fedb`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:29:03,111::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:29:03,111::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000714586', 'lastCheck': '2.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:29:03,111::task::1185::TaskManager.Task::(prepare) Task=`7b58c4ab-d546-485b-8144-0ba5bd02fedb`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000714586', 'lastCheck': '2.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:29:03,111::task::595::TaskManager.Task::(_updateState) Task=`7b58c4ab-d546-485b-8144-0ba5bd02fedb`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:29:03,112::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:29:03,112::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:29:03,112::task::990::TaskManager.Task::(_decref) Task=`7b58c4ab-d546-485b-8144-0ba5bd02fedb`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:29:03,487::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:03,567::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0380319 s, 26.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:29:03,728::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:29:03,728::task::595::TaskManager.Task::(_updateState) Task=`1f60cc90-fd71-41c3-b97c-57cd73863463`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:29:03,728::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:29:03,729::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:29:03,729::task::1185::TaskManager.Task::(prepare) Task=`1f60cc90-fd71-41c3-b97c-57cd73863463`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:29:03,729::task::595::TaskManager.Task::(_updateState) Task=`1f60cc90-fd71-41c3-b97c-57cd73863463`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:29:03,729::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:29:03,729::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:29:03,730::task::990::TaskManager.Task::(_decref) Task=`1f60cc90-fd71-41c3-b97c-57cd73863463`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:29:03,734::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:29:03,735::task::595::TaskManager.Task::(_updateState) Task=`dd62f652-af8d-4d40-bb76-a8089ccce682`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:29:03,735::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:29:03,735::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6097044c-b3c1-4625-bb13-af5aa9458691`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:29:03,736::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:29:03,736::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:29:03,736::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6097044c-b3c1-4625-bb13-af5aa9458691`::Granted request >Thread-338::DEBUG::2014-05-13 12:29:03,736::task::827::TaskManager.Task::(resourceAcquired) Task=`dd62f652-af8d-4d40-bb76-a8089ccce682`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:29:03,737::task::990::TaskManager.Task::(_decref) Task=`dd62f652-af8d-4d40-bb76-a8089ccce682`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:29:03,737::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:29:03,738::task::1185::TaskManager.Task::(prepare) Task=`dd62f652-af8d-4d40-bb76-a8089ccce682`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:29:03,738::task::595::TaskManager.Task::(_updateState) Task=`dd62f652-af8d-4d40-bb76-a8089ccce682`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:29:03,738::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:29:03,738::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:29:03,738::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:29:03,739::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:29:03,739::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:29:03,739::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:29:03,739::task::990::TaskManager.Task::(_decref) Task=`dd62f652-af8d-4d40-bb76-a8089ccce682`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:29:04,186::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:29:05,580::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:05,655::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.032986 s, 31.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:29:07,205::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:29:07,668::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:07,757::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0456185 s, 22.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:29:09,770::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:09,846::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0340728 s, 30.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:29:10,184::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:29:10,192::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000826185 s, 5.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:29:10,223::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:29:11,859::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:11,934::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0329338 s, 31.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:29:13,242::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:29:13,755::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [19876d15] >Thread-13::DEBUG::2014-05-13 12:29:13,756::task::595::TaskManager.Task::(_updateState) Task=`73d3dfc6-c72c-4a85-ad8a-b1b0df30f2af`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:29:13,756::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:29:13,757::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:29:13,757::task::1185::TaskManager.Task::(prepare) Task=`73d3dfc6-c72c-4a85-ad8a-b1b0df30f2af`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:29:13,757::task::595::TaskManager.Task::(_updateState) Task=`73d3dfc6-c72c-4a85-ad8a-b1b0df30f2af`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:29:13,757::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:29:13,757::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:29:13,757::task::990::TaskManager.Task::(_decref) Task=`73d3dfc6-c72c-4a85-ad8a-b1b0df30f2af`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:29:13,762::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [19876d15] >Thread-338::DEBUG::2014-05-13 12:29:13,762::task::595::TaskManager.Task::(_updateState) Task=`9c8695e8-9c10-4144-88e7-5da56d284a94`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:29:13,762::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:29:13,763::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`89243a9b-ef4e-4300-954b-3680e6bcb304`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:29:13,763::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:29:13,763::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:29:13,763::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`89243a9b-ef4e-4300-954b-3680e6bcb304`::Granted request >Thread-338::DEBUG::2014-05-13 12:29:13,764::task::827::TaskManager.Task::(resourceAcquired) Task=`9c8695e8-9c10-4144-88e7-5da56d284a94`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:29:13,764::task::990::TaskManager.Task::(_decref) Task=`9c8695e8-9c10-4144-88e7-5da56d284a94`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:29:13,765::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:29:13,765::task::1185::TaskManager.Task::(prepare) Task=`9c8695e8-9c10-4144-88e7-5da56d284a94`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:29:13,765::task::595::TaskManager.Task::(_updateState) Task=`9c8695e8-9c10-4144-88e7-5da56d284a94`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:29:13,765::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:29:13,765::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:29:13,766::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:29:13,766::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:29:13,766::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:29:13,766::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:29:13,766::task::990::TaskManager.Task::(_decref) Task=`9c8695e8-9c10-4144-88e7-5da56d284a94`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:29:13,947::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:14,039::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0503883 s, 20.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:29:16,052::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:16,128::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0335587 s, 30.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:29:16,261::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:29:18,141::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:18,222::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382066 s, 26.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:29:18,245::task::595::TaskManager.Task::(_updateState) Task=`734ef7c6-4182-48a7-a707-70ec9dbae5b7`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:29:18,246::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:29:18,246::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000826185', 'lastCheck': '8.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:29:18,246::task::1185::TaskManager.Task::(prepare) Task=`734ef7c6-4182-48a7-a707-70ec9dbae5b7`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000826185', 'lastCheck': '8.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:29:18,246::task::595::TaskManager.Task::(_updateState) Task=`734ef7c6-4182-48a7-a707-70ec9dbae5b7`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:29:18,246::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:29:18,247::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:29:18,247::task::990::TaskManager.Task::(_decref) Task=`734ef7c6-4182-48a7-a707-70ec9dbae5b7`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:29:19,279::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:29:20,199::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:29:20,207::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.0006012 s, 6.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:29:20,234::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:20,312::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359424 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:29:22,297::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:29:22,325::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:22,401::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031809 s, 32.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:29:23,784::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:29:23,785::task::595::TaskManager.Task::(_updateState) Task=`093a7750-a749-4012-89c1-ef78f3569a97`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:29:23,785::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:29:23,785::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:29:23,785::task::1185::TaskManager.Task::(prepare) Task=`093a7750-a749-4012-89c1-ef78f3569a97`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:29:23,786::task::595::TaskManager.Task::(_updateState) Task=`093a7750-a749-4012-89c1-ef78f3569a97`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:29:23,786::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:29:23,786::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:29:23,786::task::990::TaskManager.Task::(_decref) Task=`093a7750-a749-4012-89c1-ef78f3569a97`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:29:23,791::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:29:23,791::task::595::TaskManager.Task::(_updateState) Task=`e9165eb7-115a-4b23-9b70-e98dd2bebd25`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:29:23,791::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:29:23,792::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0139089e-8744-4170-9914-1df2d822c6aa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:29:23,792::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:29:23,792::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:29:23,792::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0139089e-8744-4170-9914-1df2d822c6aa`::Granted request >Thread-338::DEBUG::2014-05-13 12:29:23,793::task::827::TaskManager.Task::(resourceAcquired) Task=`e9165eb7-115a-4b23-9b70-e98dd2bebd25`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:29:23,793::task::990::TaskManager.Task::(_decref) Task=`e9165eb7-115a-4b23-9b70-e98dd2bebd25`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:29:23,793::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:29:23,794::task::1185::TaskManager.Task::(prepare) Task=`e9165eb7-115a-4b23-9b70-e98dd2bebd25`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:29:23,794::task::595::TaskManager.Task::(_updateState) Task=`e9165eb7-115a-4b23-9b70-e98dd2bebd25`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:29:23,794::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:29:23,794::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:29:23,794::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:29:23,794::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:29:23,795::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:29:23,795::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:29:23,795::task::990::TaskManager.Task::(_decref) Task=`e9165eb7-115a-4b23-9b70-e98dd2bebd25`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:29:24,414::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:24,503::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318298 s, 32.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:29:25,315::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:29:26,516::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:26,592::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383635 s, 26.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:29:28,335::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:29:28,600::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:28,693::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0493452 s, 20.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:29:30,214::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:29:30,221::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000627996 s, 6.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:29:30,706::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:30,785::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036211 s, 28.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:29:31,353::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:29:32,798::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:32,882::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0391014 s, 26.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:29:33,379::task::595::TaskManager.Task::(_updateState) Task=`740c773c-ca88-4307-95b2-21069af1faa7`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:29:33,379::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:29:33,380::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000627996', 'lastCheck': '3.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:29:33,380::task::1185::TaskManager.Task::(prepare) Task=`740c773c-ca88-4307-95b2-21069af1faa7`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000627996', 'lastCheck': '3.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:29:33,380::task::595::TaskManager.Task::(_updateState) Task=`740c773c-ca88-4307-95b2-21069af1faa7`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:29:33,380::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:29:33,380::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:29:33,380::task::990::TaskManager.Task::(_decref) Task=`740c773c-ca88-4307-95b2-21069af1faa7`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 12:29:33,815::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:29:33,815::task::595::TaskManager.Task::(_updateState) Task=`1d0cb174-3565-409c-9ae1-0553dfb77769`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:29:33,816::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:29:33,816::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:29:33,816::task::1185::TaskManager.Task::(prepare) Task=`1d0cb174-3565-409c-9ae1-0553dfb77769`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:29:33,816::task::595::TaskManager.Task::(_updateState) Task=`1d0cb174-3565-409c-9ae1-0553dfb77769`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:29:33,817::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:29:33,817::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:29:33,817::task::990::TaskManager.Task::(_decref) Task=`1d0cb174-3565-409c-9ae1-0553dfb77769`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:29:33,821::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:29:33,822::task::595::TaskManager.Task::(_updateState) Task=`4a51ed0a-ade9-4ac8-9d8d-8d18ab2e5578`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:29:33,822::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:29:33,822::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`205f2174-2112-409e-8c35-f8d4f73b49a4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:29:33,822::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:29:33,823::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:29:33,823::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`205f2174-2112-409e-8c35-f8d4f73b49a4`::Granted request >Thread-338::DEBUG::2014-05-13 12:29:33,823::task::827::TaskManager.Task::(resourceAcquired) Task=`4a51ed0a-ade9-4ac8-9d8d-8d18ab2e5578`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:29:33,823::task::990::TaskManager.Task::(_decref) Task=`4a51ed0a-ade9-4ac8-9d8d-8d18ab2e5578`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:29:33,824::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:29:33,824::task::1185::TaskManager.Task::(prepare) Task=`4a51ed0a-ade9-4ac8-9d8d-8d18ab2e5578`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:29:33,824::task::595::TaskManager.Task::(_updateState) Task=`4a51ed0a-ade9-4ac8-9d8d-8d18ab2e5578`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:29:33,825::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:29:33,825::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:29:33,825::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:29:33,825::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:29:33,825::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:29:33,825::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:29:33,826::task::990::TaskManager.Task::(_decref) Task=`4a51ed0a-ade9-4ac8-9d8d-8d18ab2e5578`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:29:34,371::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:29:34,896::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:34,978::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383651 s, 26.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:29:36,991::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:37,065::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313657 s, 32.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:29:37,390::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:29:39,078::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:39,154::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321526 s, 31.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:29:40,229::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:29:40,236::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000638055 s, 6.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:29:40,412::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:29:41,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:41,584::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.374663 s, 2.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:29:43,431::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:29:43,595::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:43,678::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384721 s, 26.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:29:43,843::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [dc1f215] >Thread-13::DEBUG::2014-05-13 12:29:43,844::task::595::TaskManager.Task::(_updateState) Task=`75940d3d-faba-4f60-8cb1-1e8dc750437b`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:29:43,844::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:29:43,844::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:29:43,845::task::1185::TaskManager.Task::(prepare) Task=`75940d3d-faba-4f60-8cb1-1e8dc750437b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:29:43,845::task::595::TaskManager.Task::(_updateState) Task=`75940d3d-faba-4f60-8cb1-1e8dc750437b`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:29:43,845::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:29:43,845::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:29:43,845::task::990::TaskManager.Task::(_decref) Task=`75940d3d-faba-4f60-8cb1-1e8dc750437b`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:29:43,850::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [dc1f215] >Thread-338::DEBUG::2014-05-13 12:29:43,850::task::595::TaskManager.Task::(_updateState) Task=`cfae3e41-a202-4442-a784-88a596964fda`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:29:43,851::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:29:43,851::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4ff9f065-037e-4af9-b4e1-4136b3b4a166`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:29:43,851::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:29:43,852::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:29:43,852::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4ff9f065-037e-4af9-b4e1-4136b3b4a166`::Granted request >Thread-338::DEBUG::2014-05-13 12:29:43,852::task::827::TaskManager.Task::(resourceAcquired) Task=`cfae3e41-a202-4442-a784-88a596964fda`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:29:43,852::task::990::TaskManager.Task::(_decref) Task=`cfae3e41-a202-4442-a784-88a596964fda`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:29:43,853::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:29:43,853::task::1185::TaskManager.Task::(prepare) Task=`cfae3e41-a202-4442-a784-88a596964fda`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:29:43,853::task::595::TaskManager.Task::(_updateState) Task=`cfae3e41-a202-4442-a784-88a596964fda`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:29:43,854::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:29:43,854::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:29:43,854::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:29:43,854::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:29:43,854::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:29:43,854::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:29:43,855::task::990::TaskManager.Task::(_decref) Task=`cfae3e41-a202-4442-a784-88a596964fda`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:29:45,690::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:45,766::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320338 s, 32.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:29:46,448::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:29:47,781::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:47,857::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0329352 s, 31.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:29:48,520::task::595::TaskManager.Task::(_updateState) Task=`87d55cb1-8990-4761-b7a2-3f7b15439669`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:29:48,520::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:29:48,521::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000638055', 'lastCheck': '8.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:29:48,521::task::1185::TaskManager.Task::(prepare) Task=`87d55cb1-8990-4761-b7a2-3f7b15439669`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000638055', 'lastCheck': '8.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:29:48,521::task::595::TaskManager.Task::(_updateState) Task=`87d55cb1-8990-4761-b7a2-3f7b15439669`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:29:48,521::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:29:48,521::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:29:48,521::task::990::TaskManager.Task::(_decref) Task=`87d55cb1-8990-4761-b7a2-3f7b15439669`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:29:49,467::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:29:49,869::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:49,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0380548 s, 26.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:29:50,244::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:29:50,252::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000638672 s, 6.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:29:51,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:52,040::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319841 s, 32.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:29:52,487::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:29:53,874::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:29:53,875::task::595::TaskManager.Task::(_updateState) Task=`9a3c4c80-5447-4299-a0ce-25a76b1d7665`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:29:53,875::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:29:53,875::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:29:53,875::task::1185::TaskManager.Task::(prepare) Task=`9a3c4c80-5447-4299-a0ce-25a76b1d7665`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:29:53,876::task::595::TaskManager.Task::(_updateState) Task=`9a3c4c80-5447-4299-a0ce-25a76b1d7665`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:29:53,876::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:29:53,876::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:29:53,876::task::990::TaskManager.Task::(_decref) Task=`9a3c4c80-5447-4299-a0ce-25a76b1d7665`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:29:53,881::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:29:53,881::task::595::TaskManager.Task::(_updateState) Task=`f572c445-b50f-400e-8982-c64dab27bd46`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:29:53,881::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:29:53,882::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e1b27c11-fefb-47f2-a391-2cabd7fe2ca1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:29:53,882::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:29:53,882::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:29:53,882::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e1b27c11-fefb-47f2-a391-2cabd7fe2ca1`::Granted request >Thread-338::DEBUG::2014-05-13 12:29:53,882::task::827::TaskManager.Task::(resourceAcquired) Task=`f572c445-b50f-400e-8982-c64dab27bd46`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:29:53,883::task::990::TaskManager.Task::(_decref) Task=`f572c445-b50f-400e-8982-c64dab27bd46`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:29:53,883::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:29:53,883::task::1185::TaskManager.Task::(prepare) Task=`f572c445-b50f-400e-8982-c64dab27bd46`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:29:53,884::task::595::TaskManager.Task::(_updateState) Task=`f572c445-b50f-400e-8982-c64dab27bd46`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:29:53,884::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:29:53,884::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:29:53,884::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:29:53,884::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:29:53,884::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:29:53,884::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:29:53,885::task::990::TaskManager.Task::(_decref) Task=`f572c445-b50f-400e-8982-c64dab27bd46`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:29:54,053::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:54,127::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305768 s, 33.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:29:55,506::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:29:56,140::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:56,230::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.047289 s, 21.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:29:58,243::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:29:58,322::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0354224 s, 28.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:29:58,526::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:30:00,258::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:30:00,266::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000621602 s, 6.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:30:00,335::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:30:00,414::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363723 s, 28.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:30:01,547::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:30:02,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:30:02,506::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364872 s, 28.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:30:03,664::task::595::TaskManager.Task::(_updateState) Task=`f3b9552e-343b-46cc-9a87-7cf60ae6ee0b`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:30:03,664::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:30:03,664::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000621602', 'lastCheck': '3.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:30:03,664::task::1185::TaskManager.Task::(prepare) Task=`f3b9552e-343b-46cc-9a87-7cf60ae6ee0b`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000621602', 'lastCheck': '3.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:30:03,664::task::595::TaskManager.Task::(_updateState) Task=`f3b9552e-343b-46cc-9a87-7cf60ae6ee0b`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:30:03,665::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:30:03,665::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:30:03,665::task::990::TaskManager.Task::(_decref) Task=`f3b9552e-343b-46cc-9a87-7cf60ae6ee0b`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 12:30:03,905::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6602059] >Thread-13::DEBUG::2014-05-13 12:30:03,905::task::595::TaskManager.Task::(_updateState) Task=`cde21cc3-0c42-4745-b92a-008a4594d9bb`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:30:03,906::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:30:03,906::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:30:03,906::task::1185::TaskManager.Task::(prepare) Task=`cde21cc3-0c42-4745-b92a-008a4594d9bb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:30:03,906::task::595::TaskManager.Task::(_updateState) Task=`cde21cc3-0c42-4745-b92a-008a4594d9bb`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:30:03,907::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:30:03,907::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:30:03,907::task::990::TaskManager.Task::(_decref) Task=`cde21cc3-0c42-4745-b92a-008a4594d9bb`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:30:03,911::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6602059] >Thread-338::DEBUG::2014-05-13 12:30:03,912::task::595::TaskManager.Task::(_updateState) Task=`c218a759-6c25-431d-9679-ea358d300f33`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:30:03,912::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:30:03,912::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d0f04c04-bf2f-4af8-a88e-3fbbe3726085`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:30:03,912::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:30:03,913::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:30:03,913::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d0f04c04-bf2f-4af8-a88e-3fbbe3726085`::Granted request >Thread-338::DEBUG::2014-05-13 12:30:03,913::task::827::TaskManager.Task::(resourceAcquired) Task=`c218a759-6c25-431d-9679-ea358d300f33`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:30:03,913::task::990::TaskManager.Task::(_decref) Task=`c218a759-6c25-431d-9679-ea358d300f33`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:30:03,914::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:30:03,914::task::1185::TaskManager.Task::(prepare) Task=`c218a759-6c25-431d-9679-ea358d300f33`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:30:03,914::task::595::TaskManager.Task::(_updateState) Task=`c218a759-6c25-431d-9679-ea358d300f33`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:30:03,915::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:30:03,915::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:30:03,915::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:30:03,915::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:30:03,915::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:30:03,916::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:30:03,916::task::990::TaskManager.Task::(_decref) Task=`c218a759-6c25-431d-9679-ea358d300f33`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:30:04,519::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:30:04,566::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:30:04,619::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.038446 s, 26.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:30:06,632::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:30:06,705::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306425 s, 33.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:30:07,584::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:30:08,718::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:30:08,791::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305914 s, 33.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:30:10,271::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 41830dad-f012-4b4d-b636-863f66d5ca4b >Thread-322::DEBUG::2014-05-13 12:30:10,271::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-322::DEBUG::2014-05-13 12:30:10,274::blockSD::336::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:30:10,283::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.00184317 s, 1.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:30:10,283::misc::287::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.00184317 s, 1.1 MB/s'], size: 2048 >Thread-322::DEBUG::2014-05-13 12:30:10,283::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-322::WARNING::2014-05-13 12:30:10,284::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-322::DEBUG::2014-05-13 12:30:10,284::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-322::DEBUG::2014-05-13 12:30:10,284::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:30:10,285::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:30:10,285::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:30:10,285::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:30:10,285::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:30:10,286::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 12:30:10,551::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:30:10,553::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:30:10,553::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'LOCKPOLICY=', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', 'DESCRIPTION=ISCSI', 'LOGBLKSIZE=512', 'TYPE=ISCSI', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'PHYBLKSIZE=512', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', 'CLASS=Data', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'POOL_DESCRIPTION=DC_2', 'POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'MASTER_VERSION=1', 'ROLE=Master', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', '_SHA_CKSUM=28bb2d7cc60e0d779eb69fe93d050c16c769f069', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0'] >Thread-322::INFO::2014-05-13 12:30:10,555::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_imageNS already registered >Thread-322::INFO::2014-05-13 12:30:10,555::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_volumeNS already registered >Thread-322::INFO::2014-05-13 12:30:10,555::blockSD::452::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_lvmActivationNS already registered >Thread-322::DEBUG::2014-05-13 12:30:10,556::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:30:10,603::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:30:10,769::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:30:10,772::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:30:10,780::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000517932 s, 7.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:30:10,804::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:30:11,175::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.327035 s, 3.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:30:13,189::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:30:13,272::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384362 s, 26.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:30:13,634::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:30:13,934::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [28fd42b9] >Thread-13::DEBUG::2014-05-13 12:30:13,934::task::595::TaskManager.Task::(_updateState) Task=`197acc04-deb0-4ce4-b6eb-262f86878b12`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:30:13,934::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:30:13,935::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:30:13,935::task::1185::TaskManager.Task::(prepare) Task=`197acc04-deb0-4ce4-b6eb-262f86878b12`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:30:13,935::task::595::TaskManager.Task::(_updateState) Task=`197acc04-deb0-4ce4-b6eb-262f86878b12`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:30:13,935::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:30:13,935::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:30:13,936::task::990::TaskManager.Task::(_decref) Task=`197acc04-deb0-4ce4-b6eb-262f86878b12`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:30:13,940::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [28fd42b9] >Thread-338::DEBUG::2014-05-13 12:30:13,940::task::595::TaskManager.Task::(_updateState) Task=`e2d9b181-a00c-42ac-9744-24c5920cb3ba`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:30:13,940::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:30:13,941::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a5736254-d184-470c-83fb-09a16ee775b9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:30:13,941::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:30:13,941::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:30:13,941::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a5736254-d184-470c-83fb-09a16ee775b9`::Granted request >Thread-338::DEBUG::2014-05-13 12:30:13,942::task::827::TaskManager.Task::(resourceAcquired) Task=`e2d9b181-a00c-42ac-9744-24c5920cb3ba`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:30:13,942::task::990::TaskManager.Task::(_decref) Task=`e2d9b181-a00c-42ac-9744-24c5920cb3ba`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:30:13,943::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:30:13,943::task::1185::TaskManager.Task::(prepare) Task=`e2d9b181-a00c-42ac-9744-24c5920cb3ba`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:30:13,943::task::595::TaskManager.Task::(_updateState) Task=`e2d9b181-a00c-42ac-9744-24c5920cb3ba`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:30:13,943::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:30:13,943::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:30:13,944::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:30:13,944::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:30:13,944::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:30:13,944::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:30:13,944::task::990::TaskManager.Task::(_decref) Task=`e2d9b181-a00c-42ac-9744-24c5920cb3ba`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:30:15,285::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:30:15,359::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309166 s, 33.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:30:16,654::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:30:17,372::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:30:17,452::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.038663 s, 26.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:30:18,800::task::595::TaskManager.Task::(_updateState) Task=`e261193e-d476-4160-aca7-2dbd19cda706`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:30:18,801::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:30:18,801::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000517932', 'lastCheck': '8.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:30:18,801::task::1185::TaskManager.Task::(prepare) Task=`e261193e-d476-4160-aca7-2dbd19cda706`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000517932', 'lastCheck': '8.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:30:18,801::task::595::TaskManager.Task::(_updateState) Task=`e261193e-d476-4160-aca7-2dbd19cda706`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:30:18,801::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:30:18,801::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:30:18,802::task::990::TaskManager.Task::(_decref) Task=`e261193e-d476-4160-aca7-2dbd19cda706`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:30:19,465::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:30:19,546::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0389115 s, 26.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:30:19,674::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:30:20,788::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:30:20,796::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00106031 s, 3.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:30:21,559::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:30:21,962::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.360551 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:30:22,692::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:30:23,963::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:30:23,973::task::595::TaskManager.Task::(_updateState) Task=`744124d3-48df-4236-bf7d-ec4ddccfad64`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:30:23,973::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:30:23,973::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:30:23,973::task::1185::TaskManager.Task::(prepare) Task=`744124d3-48df-4236-bf7d-ec4ddccfad64`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:30:23,974::task::595::TaskManager.Task::(_updateState) Task=`744124d3-48df-4236-bf7d-ec4ddccfad64`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:30:23,974::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:30:23,974::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:30:23,974::task::990::TaskManager.Task::(_decref) Task=`744124d3-48df-4236-bf7d-ec4ddccfad64`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:30:23,975::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-338::DEBUG::2014-05-13 12:30:23,987::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:30:23,987::task::595::TaskManager.Task::(_updateState) Task=`56adce97-c75a-4432-b2a4-538797c60b90`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:30:23,988::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:30:23,988::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`36d848c1-a7a0-4af6-a583-6fb43cb7cb7e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:30:23,988::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:30:23,989::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:30:23,989::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`36d848c1-a7a0-4af6-a583-6fb43cb7cb7e`::Granted request >Thread-338::DEBUG::2014-05-13 12:30:23,989::task::827::TaskManager.Task::(resourceAcquired) Task=`56adce97-c75a-4432-b2a4-538797c60b90`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:30:23,990::task::990::TaskManager.Task::(_decref) Task=`56adce97-c75a-4432-b2a4-538797c60b90`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:30:23,990::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:30:23,991::task::1185::TaskManager.Task::(prepare) Task=`56adce97-c75a-4432-b2a4-538797c60b90`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:30:23,991::task::595::TaskManager.Task::(_updateState) Task=`56adce97-c75a-4432-b2a4-538797c60b90`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:30:23,991::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:30:23,991::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:30:23,992::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:30:23,992::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:30:23,992::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:30:23,993::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:30:23,993::task::990::TaskManager.Task::(_decref) Task=`56adce97-c75a-4432-b2a4-538797c60b90`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:30:24,052::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0342698 s, 29.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:30:25,712::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:30:26,065::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:30:26,144::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355984 s, 28.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:30:28,157::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:30:28,239::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0371842 s, 27.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:30:28,730::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:30:30,252::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:30:30,326::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0308513 s, 33.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:30:30,803::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:30:30,811::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000624641 s, 6.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:30:31,749::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:30:32,339::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:30:32,420::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0371204 s, 27.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:30:33,942::task::595::TaskManager.Task::(_updateState) Task=`ba979608-fc7e-4d51-9c1f-aa3188d6dc3e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:30:33,943::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:30:33,943::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000624641', 'lastCheck': '3.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:30:33,943::task::1185::TaskManager.Task::(prepare) Task=`ba979608-fc7e-4d51-9c1f-aa3188d6dc3e`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000624641', 'lastCheck': '3.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:30:33,943::task::595::TaskManager.Task::(_updateState) Task=`ba979608-fc7e-4d51-9c1f-aa3188d6dc3e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:30:33,943::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:30:33,943::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:30:33,944::task::990::TaskManager.Task::(_decref) Task=`ba979608-fc7e-4d51-9c1f-aa3188d6dc3e`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 12:30:34,013::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:30:34,014::task::595::TaskManager.Task::(_updateState) Task=`85a21607-44ab-4e46-8743-5763531a40b8`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:30:34,014::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:30:34,014::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:30:34,014::task::1185::TaskManager.Task::(prepare) Task=`85a21607-44ab-4e46-8743-5763531a40b8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:30:34,015::task::595::TaskManager.Task::(_updateState) Task=`85a21607-44ab-4e46-8743-5763531a40b8`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:30:34,015::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:30:34,015::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:30:34,015::task::990::TaskManager.Task::(_decref) Task=`85a21607-44ab-4e46-8743-5763531a40b8`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:30:34,060::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:30:34,060::task::595::TaskManager.Task::(_updateState) Task=`ab00ecdd-a45e-41c6-8ab4-3dcbe1dfe72a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:30:34,060::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:30:34,061::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8c469e72-8be4-4ab8-83e6-1fa7ce8614c3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:30:34,061::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:30:34,061::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:30:34,061::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8c469e72-8be4-4ab8-83e6-1fa7ce8614c3`::Granted request >Thread-338::DEBUG::2014-05-13 12:30:34,062::task::827::TaskManager.Task::(resourceAcquired) Task=`ab00ecdd-a45e-41c6-8ab4-3dcbe1dfe72a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:30:34,062::task::990::TaskManager.Task::(_decref) Task=`ab00ecdd-a45e-41c6-8ab4-3dcbe1dfe72a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:30:34,063::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:30:34,063::task::1185::TaskManager.Task::(prepare) Task=`ab00ecdd-a45e-41c6-8ab4-3dcbe1dfe72a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:30:34,063::task::595::TaskManager.Task::(_updateState) Task=`ab00ecdd-a45e-41c6-8ab4-3dcbe1dfe72a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:30:34,063::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:30:34,063::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:30:34,064::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:30:34,064::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:30:34,064::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:30:34,064::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:30:34,064::task::990::TaskManager.Task::(_decref) Task=`ab00ecdd-a45e-41c6-8ab4-3dcbe1dfe72a`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:30:34,433::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:30:34,760::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0368046 s, 27.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:30:34,766::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:30:36,782::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:30:37,145::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.320198 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:30:37,784::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:30:39,158::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:30:39,234::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316584 s, 32.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:30:40,803::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:30:40,818::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:30:40,826::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000559701 s, 7.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:30:41,247::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:30:41,327::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0368222 s, 27.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:30:43,340::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:30:43,421::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0370341 s, 27.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:30:43,820::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:30:44,083::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1ea398d3] >Thread-13::DEBUG::2014-05-13 12:30:44,083::task::595::TaskManager.Task::(_updateState) Task=`2b629b58-caab-482f-8e0a-9b6b50a52bea`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:30:44,084::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:30:44,084::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:30:44,084::task::1185::TaskManager.Task::(prepare) Task=`2b629b58-caab-482f-8e0a-9b6b50a52bea`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:30:44,084::task::595::TaskManager.Task::(_updateState) Task=`2b629b58-caab-482f-8e0a-9b6b50a52bea`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:30:44,085::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:30:44,085::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:30:44,085::task::990::TaskManager.Task::(_decref) Task=`2b629b58-caab-482f-8e0a-9b6b50a52bea`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:30:44,089::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1ea398d3] >Thread-338::DEBUG::2014-05-13 12:30:44,089::task::595::TaskManager.Task::(_updateState) Task=`dac2f613-ae4f-496a-8f31-5003e3de0015`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:30:44,090::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:30:44,090::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9bf23d36-2552-4098-a9e7-dfdc0aeeff80`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:30:44,090::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:30:44,090::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:30:44,091::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9bf23d36-2552-4098-a9e7-dfdc0aeeff80`::Granted request >Thread-338::DEBUG::2014-05-13 12:30:44,091::task::827::TaskManager.Task::(resourceAcquired) Task=`dac2f613-ae4f-496a-8f31-5003e3de0015`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:30:44,091::task::990::TaskManager.Task::(_decref) Task=`dac2f613-ae4f-496a-8f31-5003e3de0015`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:30:44,092::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:30:44,092::task::1185::TaskManager.Task::(prepare) Task=`dac2f613-ae4f-496a-8f31-5003e3de0015`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:30:44,092::task::595::TaskManager.Task::(_updateState) Task=`dac2f613-ae4f-496a-8f31-5003e3de0015`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:30:44,092::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:30:44,093::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:30:44,093::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:30:44,093::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:30:44,093::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:30:44,093::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:30:44,094::task::990::TaskManager.Task::(_decref) Task=`dac2f613-ae4f-496a-8f31-5003e3de0015`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:30:45,434::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:30:45,510::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317423 s, 32.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:30:46,845::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:30:47,523::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:30:47,604::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0388718 s, 26.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:30:49,085::task::595::TaskManager.Task::(_updateState) Task=`46bf4194-2086-4a64-b9f8-6d1bc45d9318`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:30:49,085::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:30:49,086::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000559701', 'lastCheck': '8.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:30:49,086::task::1185::TaskManager.Task::(prepare) Task=`46bf4194-2086-4a64-b9f8-6d1bc45d9318`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000559701', 'lastCheck': '8.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:30:49,086::task::595::TaskManager.Task::(_updateState) Task=`46bf4194-2086-4a64-b9f8-6d1bc45d9318`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:30:49,086::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:30:49,086::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:30:49,086::task::990::TaskManager.Task::(_decref) Task=`46bf4194-2086-4a64-b9f8-6d1bc45d9318`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:30:49,617::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:30:49,863::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:30:49,982::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.324011 s, 3.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:30:50,833::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:30:50,841::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000677844 s, 6.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:30:51,995::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:30:52,076::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0369358 s, 27.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:30:52,881::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:30:54,089::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 12:30:54,111::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:30:54,112::task::595::TaskManager.Task::(_updateState) Task=`7d5c363b-68cb-4aba-ab87-d0fad1a6f3f1`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:30:54,112::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:30:54,113::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:30:54,113::task::1185::TaskManager.Task::(prepare) Task=`7d5c363b-68cb-4aba-ab87-d0fad1a6f3f1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:30:54,113::task::595::TaskManager.Task::(_updateState) Task=`7d5c363b-68cb-4aba-ab87-d0fad1a6f3f1`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:30:54,114::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:30:54,114::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:30:54,114::task::990::TaskManager.Task::(_decref) Task=`7d5c363b-68cb-4aba-ab87-d0fad1a6f3f1`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:30:54,120::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:30:54,120::task::595::TaskManager.Task::(_updateState) Task=`a5e2b13b-c164-4ce7-bc78-dbae80655d39`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:30:54,120::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:30:54,121::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`235bdb04-4c4e-4d80-9153-bcbab6cbf40d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:30:54,121::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:30:54,121::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:30:54,122::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`235bdb04-4c4e-4d80-9153-bcbab6cbf40d`::Granted request >Thread-338::DEBUG::2014-05-13 12:30:54,122::task::827::TaskManager.Task::(resourceAcquired) Task=`a5e2b13b-c164-4ce7-bc78-dbae80655d39`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:30:54,122::task::990::TaskManager.Task::(_decref) Task=`a5e2b13b-c164-4ce7-bc78-dbae80655d39`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:30:54,123::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:30:54,124::task::1185::TaskManager.Task::(prepare) Task=`a5e2b13b-c164-4ce7-bc78-dbae80655d39`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:30:54,124::task::595::TaskManager.Task::(_updateState) Task=`a5e2b13b-c164-4ce7-bc78-dbae80655d39`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:30:54,124::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:30:54,124::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:30:54,125::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:30:54,125::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:30:54,125::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:30:54,125::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:30:54,126::task::990::TaskManager.Task::(_decref) Task=`a5e2b13b-c164-4ce7-bc78-dbae80655d39`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:30:54,176::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0430174 s, 23.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:30:55,899::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:30:56,190::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:30:56,270::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384883 s, 26.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:30:58,283::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:30:58,365::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0369743 s, 27.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:30:58,916::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:31:00,377::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:31:00,735::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.315226 s, 3.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:31:00,848::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:31:00,856::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000828395 s, 4.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:31:01,934::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:31:02,748::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:31:02,830::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0388392 s, 26.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:31:04,147::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:31:04,147::task::595::TaskManager.Task::(_updateState) Task=`a15bacb3-a1b7-4081-8cb5-e4d7024ee5a0`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:31:04,147::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:31:04,148::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:31:04,148::task::1185::TaskManager.Task::(prepare) Task=`a15bacb3-a1b7-4081-8cb5-e4d7024ee5a0`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:31:04,148::task::595::TaskManager.Task::(_updateState) Task=`a15bacb3-a1b7-4081-8cb5-e4d7024ee5a0`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:31:04,148::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:31:04,149::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:31:04,149::task::990::TaskManager.Task::(_decref) Task=`a15bacb3-a1b7-4081-8cb5-e4d7024ee5a0`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:31:04,153::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:31:04,153::task::595::TaskManager.Task::(_updateState) Task=`3da57978-01f2-484d-8af4-0323ee112a31`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:31:04,154::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:31:04,154::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2512a3d5-2c7b-42f4-b342-41a7ca725719`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:31:04,154::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:31:04,155::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:31:04,155::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2512a3d5-2c7b-42f4-b342-41a7ca725719`::Granted request >Thread-338::DEBUG::2014-05-13 12:31:04,155::task::827::TaskManager.Task::(resourceAcquired) Task=`3da57978-01f2-484d-8af4-0323ee112a31`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:31:04,155::task::990::TaskManager.Task::(_decref) Task=`3da57978-01f2-484d-8af4-0323ee112a31`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:31:04,156::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:31:04,156::task::1185::TaskManager.Task::(prepare) Task=`3da57978-01f2-484d-8af4-0323ee112a31`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:31:04,156::task::595::TaskManager.Task::(_updateState) Task=`3da57978-01f2-484d-8af4-0323ee112a31`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:31:04,157::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:31:04,157::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:31:04,157::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:31:04,157::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:31:04,157::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:31:04,158::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:31:04,158::task::990::TaskManager.Task::(_decref) Task=`3da57978-01f2-484d-8af4-0323ee112a31`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 12:31:04,227::task::595::TaskManager.Task::(_updateState) Task=`9cd37139-1e02-46b0-9177-5623e008c0d4`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:31:04,227::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:31:04,227::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000828395', 'lastCheck': '3.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:31:04,227::task::1185::TaskManager.Task::(prepare) Task=`9cd37139-1e02-46b0-9177-5623e008c0d4`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000828395', 'lastCheck': '3.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:31:04,227::task::595::TaskManager.Task::(_updateState) Task=`9cd37139-1e02-46b0-9177-5623e008c0d4`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:31:04,228::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:31:04,228::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:31:04,228::task::990::TaskManager.Task::(_decref) Task=`9cd37139-1e02-46b0-9177-5623e008c0d4`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:31:04,844::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:31:04,954::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:31:04,986::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0971878 s, 10.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:31:06,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:31:07,074::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318483 s, 32.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:31:07,977::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:31:09,087::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:31:09,162::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319137 s, 32.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:31:10,863::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:31:10,873::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00234009 s, 1.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:31:10,994::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:31:11,175::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:31:11,257::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382636 s, 26.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:31:13,269::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:31:13,356::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0386907 s, 26.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:31:14,012::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:31:14,178::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [45b9a91f] >Thread-13::DEBUG::2014-05-13 12:31:14,178::task::595::TaskManager.Task::(_updateState) Task=`8aab79bd-3d8b-442c-8c3e-17edbb93c028`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:31:14,178::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:31:14,179::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:31:14,179::task::1185::TaskManager.Task::(prepare) Task=`8aab79bd-3d8b-442c-8c3e-17edbb93c028`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:31:14,179::task::595::TaskManager.Task::(_updateState) Task=`8aab79bd-3d8b-442c-8c3e-17edbb93c028`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:31:14,179::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:31:14,179::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:31:14,179::task::990::TaskManager.Task::(_decref) Task=`8aab79bd-3d8b-442c-8c3e-17edbb93c028`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:31:14,184::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [45b9a91f] >Thread-338::DEBUG::2014-05-13 12:31:14,184::task::595::TaskManager.Task::(_updateState) Task=`1ad938ea-b58f-48d1-a4a4-2a136f2bdb76`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:31:14,184::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:31:14,185::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`dad7df52-fe91-4dca-80c1-b4c973c92d48`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:31:14,185::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:31:14,185::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:31:14,185::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`dad7df52-fe91-4dca-80c1-b4c973c92d48`::Granted request >Thread-338::DEBUG::2014-05-13 12:31:14,185::task::827::TaskManager.Task::(resourceAcquired) Task=`1ad938ea-b58f-48d1-a4a4-2a136f2bdb76`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:31:14,186::task::990::TaskManager.Task::(_decref) Task=`1ad938ea-b58f-48d1-a4a4-2a136f2bdb76`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:31:14,186::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:31:14,187::task::1185::TaskManager.Task::(prepare) Task=`1ad938ea-b58f-48d1-a4a4-2a136f2bdb76`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:31:14,187::task::595::TaskManager.Task::(_updateState) Task=`1ad938ea-b58f-48d1-a4a4-2a136f2bdb76`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:31:14,187::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:31:14,187::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:31:14,187::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:31:14,188::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:31:14,188::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:31:14,188::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:31:14,188::task::990::TaskManager.Task::(_decref) Task=`1ad938ea-b58f-48d1-a4a4-2a136f2bdb76`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:31:15,369::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:31:15,456::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0453253 s, 22.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:31:17,030::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:31:17,469::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:31:17,547::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363132 s, 28.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:31:19,367::task::595::TaskManager.Task::(_updateState) Task=`85996487-57eb-4b73-b675-0d74f29c25e3`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:31:19,367::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:31:19,368::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00234009', 'lastCheck': '8.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:31:19,368::task::1185::TaskManager.Task::(prepare) Task=`85996487-57eb-4b73-b675-0d74f29c25e3`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00234009', 'lastCheck': '8.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:31:19,368::task::595::TaskManager.Task::(_updateState) Task=`85996487-57eb-4b73-b675-0d74f29c25e3`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:31:19,368::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:31:19,368::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:31:19,368::task::990::TaskManager.Task::(_decref) Task=`85996487-57eb-4b73-b675-0d74f29c25e3`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:31:19,560::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:31:19,639::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036021 s, 28.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:31:20,048::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:31:20,880::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:31:20,887::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000709823 s, 5.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:31:21,652::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:31:21,733::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0390252 s, 26.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:31:23,068::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:31:23,747::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:31:23,822::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306212 s, 33.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:31:24,205::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:31:24,205::task::595::TaskManager.Task::(_updateState) Task=`cafa55ed-aea6-480e-8071-db9c99986e06`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:31:24,206::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:31:24,206::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:31:24,206::task::1185::TaskManager.Task::(prepare) Task=`cafa55ed-aea6-480e-8071-db9c99986e06`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:31:24,206::task::595::TaskManager.Task::(_updateState) Task=`cafa55ed-aea6-480e-8071-db9c99986e06`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:31:24,207::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:31:24,207::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:31:24,207::task::990::TaskManager.Task::(_decref) Task=`cafa55ed-aea6-480e-8071-db9c99986e06`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:31:24,212::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:31:24,212::task::595::TaskManager.Task::(_updateState) Task=`6d64c04b-3ef5-45dd-bf03-d60ca51866ab`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:31:24,213::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:31:24,213::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e43c777c-1e30-4927-a8f9-02df41428902`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:31:24,213::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:31:24,213::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:31:24,214::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e43c777c-1e30-4927-a8f9-02df41428902`::Granted request >Thread-338::DEBUG::2014-05-13 12:31:24,214::task::827::TaskManager.Task::(resourceAcquired) Task=`6d64c04b-3ef5-45dd-bf03-d60ca51866ab`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:31:24,215::task::990::TaskManager.Task::(_decref) Task=`6d64c04b-3ef5-45dd-bf03-d60ca51866ab`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:31:24,216::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:31:24,216::task::1185::TaskManager.Task::(prepare) Task=`6d64c04b-3ef5-45dd-bf03-d60ca51866ab`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:31:24,216::task::595::TaskManager.Task::(_updateState) Task=`6d64c04b-3ef5-45dd-bf03-d60ca51866ab`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:31:24,216::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:31:24,216::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:31:24,217::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:31:24,217::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:31:24,217::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:31:24,217::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:31:24,217::task::990::TaskManager.Task::(_decref) Task=`6d64c04b-3ef5-45dd-bf03-d60ca51866ab`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:31:25,835::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:31:25,914::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381954 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:31:26,090::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:31:27,927::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:31:28,014::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0423991 s, 24.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:31:29,107::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:31:30,026::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:31:30,101::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312342 s, 32.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:31:30,929::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:31:30,936::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000610295 s, 6.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:31:32,114::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:31:32,129::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:31:32,194::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0377483 s, 27.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:31:34,207::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 12:31:34,249::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2ec76adc] >Thread-13::DEBUG::2014-05-13 12:31:34,251::task::595::TaskManager.Task::(_updateState) Task=`4508ff00-bcc3-4108-967d-7c2021098af3`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:31:34,252::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:31:34,254::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:31:34,254::task::1185::TaskManager.Task::(prepare) Task=`4508ff00-bcc3-4108-967d-7c2021098af3`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:31:34,255::task::595::TaskManager.Task::(_updateState) Task=`4508ff00-bcc3-4108-967d-7c2021098af3`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:31:34,256::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:31:34,257::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:31:34,258::task::990::TaskManager.Task::(_decref) Task=`4508ff00-bcc3-4108-967d-7c2021098af3`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:31:34,271::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2ec76adc] >Thread-338::DEBUG::2014-05-13 12:31:34,273::task::595::TaskManager.Task::(_updateState) Task=`a4040e2c-443a-4a41-8f35-37daae64d29f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:31:34,274::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:31:34,276::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`473849e4-5550-4903-b0f2-b6e849506c7e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:31:34,277::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:31:34,279::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:31:34,280::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`473849e4-5550-4903-b0f2-b6e849506c7e`::Granted request >Thread-338::DEBUG::2014-05-13 12:31:34,282::task::827::TaskManager.Task::(resourceAcquired) Task=`a4040e2c-443a-4a41-8f35-37daae64d29f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:31:34,283::task::990::TaskManager.Task::(_decref) Task=`a4040e2c-443a-4a41-8f35-37daae64d29f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:31:34,288::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:31:34,289::task::1185::TaskManager.Task::(prepare) Task=`a4040e2c-443a-4a41-8f35-37daae64d29f`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:31:34,290::task::595::TaskManager.Task::(_updateState) Task=`a4040e2c-443a-4a41-8f35-37daae64d29f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:31:34,290::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:31:34,291::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:31:34,293::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:31:34,294::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:31:34,295::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:31:34,296::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:31:34,297::task::990::TaskManager.Task::(_decref) Task=`a4040e2c-443a-4a41-8f35-37daae64d29f`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:31:34,329::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.051438 s, 19.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:31:34,499::task::595::TaskManager.Task::(_updateState) Task=`111aa80e-0655-48d0-b976-eb53e2bbeb28`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:31:34,499::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:31:34,499::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000610295', 'lastCheck': '3.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:31:34,499::task::1185::TaskManager.Task::(prepare) Task=`111aa80e-0655-48d0-b976-eb53e2bbeb28`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000610295', 'lastCheck': '3.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:31:34,499::task::595::TaskManager.Task::(_updateState) Task=`111aa80e-0655-48d0-b976-eb53e2bbeb28`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:31:34,500::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:31:34,500::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:31:34,500::task::990::TaskManager.Task::(_decref) Task=`111aa80e-0655-48d0-b976-eb53e2bbeb28`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:31:35,145::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:31:36,342::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:31:36,418::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317537 s, 32.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:31:38,166::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:31:38,431::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:31:38,512::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0376974 s, 27.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:31:40,525::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:31:40,891::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.310355 s, 3.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:31:40,944::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:31:40,952::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000550418 s, 7.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:31:41,191::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:31:42,904::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:31:42,978::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307661 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:31:44,212::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:31:44,332::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:31:44,332::task::595::TaskManager.Task::(_updateState) Task=`588ee45c-5e62-49a1-892e-8905fe47d712`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:31:44,333::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:31:44,333::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:31:44,333::task::1185::TaskManager.Task::(prepare) Task=`588ee45c-5e62-49a1-892e-8905fe47d712`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:31:44,333::task::595::TaskManager.Task::(_updateState) Task=`588ee45c-5e62-49a1-892e-8905fe47d712`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:31:44,334::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:31:44,334::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:31:44,334::task::990::TaskManager.Task::(_decref) Task=`588ee45c-5e62-49a1-892e-8905fe47d712`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:31:44,339::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:31:44,339::task::595::TaskManager.Task::(_updateState) Task=`7ab8e287-ba01-4ce6-b060-2102d6fdc0a7`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:31:44,340::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:31:44,340::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0f1d4d43-e100-48d1-a641-74278545814b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:31:44,340::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:31:44,340::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:31:44,341::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0f1d4d43-e100-48d1-a641-74278545814b`::Granted request >Thread-338::DEBUG::2014-05-13 12:31:44,341::task::827::TaskManager.Task::(resourceAcquired) Task=`7ab8e287-ba01-4ce6-b060-2102d6fdc0a7`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:31:44,341::task::990::TaskManager.Task::(_decref) Task=`7ab8e287-ba01-4ce6-b060-2102d6fdc0a7`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:31:44,342::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:31:44,342::task::1185::TaskManager.Task::(prepare) Task=`7ab8e287-ba01-4ce6-b060-2102d6fdc0a7`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:31:44,342::task::595::TaskManager.Task::(_updateState) Task=`7ab8e287-ba01-4ce6-b060-2102d6fdc0a7`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:31:44,342::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:31:44,343::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:31:44,343::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:31:44,343::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:31:44,343::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:31:44,343::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:31:44,344::task::990::TaskManager.Task::(_decref) Task=`7ab8e287-ba01-4ce6-b060-2102d6fdc0a7`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:31:44,991::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:31:45,310::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031755 s, 32.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:31:47,230::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:31:47,324::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:31:47,403::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360946 s, 28.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:31:49,416::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:31:49,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356862 s, 28.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:31:49,637::task::595::TaskManager.Task::(_updateState) Task=`0fed2a5a-55e0-4c6f-8c0a-1cca1273d22e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:31:49,637::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:31:49,638::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000550418', 'lastCheck': '8.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:31:49,638::task::1185::TaskManager.Task::(prepare) Task=`0fed2a5a-55e0-4c6f-8c0a-1cca1273d22e`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000550418', 'lastCheck': '8.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:31:49,638::task::595::TaskManager.Task::(_updateState) Task=`0fed2a5a-55e0-4c6f-8c0a-1cca1273d22e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:31:49,638::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:31:49,638::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:31:49,638::task::990::TaskManager.Task::(_decref) Task=`0fed2a5a-55e0-4c6f-8c0a-1cca1273d22e`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:31:50,249::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:31:50,958::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:31:50,966::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000794436 s, 5.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:31:51,508::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:31:51,589::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.039445 s, 26.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:31:53,266::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:31:53,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:31:53,682::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0378467 s, 27.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:31:54,370::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:31:54,409::task::595::TaskManager.Task::(_updateState) Task=`c7fda7c8-7674-45d0-869f-f51e3d35bb0f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:31:54,409::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:31:54,410::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:31:54,410::task::1185::TaskManager.Task::(prepare) Task=`c7fda7c8-7674-45d0-869f-f51e3d35bb0f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:31:54,410::task::595::TaskManager.Task::(_updateState) Task=`c7fda7c8-7674-45d0-869f-f51e3d35bb0f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:31:54,410::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:31:54,410::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:31:54,410::task::990::TaskManager.Task::(_decref) Task=`c7fda7c8-7674-45d0-869f-f51e3d35bb0f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:31:54,415::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:31:54,415::task::595::TaskManager.Task::(_updateState) Task=`723e0445-b289-481f-b936-def1d5207fce`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:31:54,416::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:31:54,417::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`99e323ff-2ebd-416c-a7ea-9fb855a9fad0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:31:54,417::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:31:54,417::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:31:54,417::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`99e323ff-2ebd-416c-a7ea-9fb855a9fad0`::Granted request >Thread-338::DEBUG::2014-05-13 12:31:54,418::task::827::TaskManager.Task::(resourceAcquired) Task=`723e0445-b289-481f-b936-def1d5207fce`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:31:54,418::task::990::TaskManager.Task::(_decref) Task=`723e0445-b289-481f-b936-def1d5207fce`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:31:54,419::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:31:54,419::task::1185::TaskManager.Task::(prepare) Task=`723e0445-b289-481f-b936-def1d5207fce`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:31:54,419::task::595::TaskManager.Task::(_updateState) Task=`723e0445-b289-481f-b936-def1d5207fce`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:31:54,419::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:31:54,419::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:31:54,419::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:31:54,420::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:31:54,420::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:31:54,420::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:31:54,420::task::990::TaskManager.Task::(_decref) Task=`723e0445-b289-481f-b936-def1d5207fce`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:31:55,695::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:31:55,783::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0468228 s, 21.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:31:56,285::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:31:57,796::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:31:57,870::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309182 s, 33.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:31:59,307::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:31:59,883::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:31:59,958::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316131 s, 32.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:32:00,973::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:32:00,981::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000705137 s, 5.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:32:01,971::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:32:02,051::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0367974 s, 27.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:32:02,327::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:32:04,064::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:32:04,145::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0375631 s, 27.3 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:32:04,439::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17df19a0] >Thread-13::DEBUG::2014-05-13 12:32:04,439::task::595::TaskManager.Task::(_updateState) Task=`2205b836-8b84-4e9f-bd63-4a5503456e1d`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:32:04,439::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:32:04,440::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:32:04,440::task::1185::TaskManager.Task::(prepare) Task=`2205b836-8b84-4e9f-bd63-4a5503456e1d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:32:04,440::task::595::TaskManager.Task::(_updateState) Task=`2205b836-8b84-4e9f-bd63-4a5503456e1d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:32:04,440::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:32:04,440::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:32:04,441::task::990::TaskManager.Task::(_decref) Task=`2205b836-8b84-4e9f-bd63-4a5503456e1d`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:32:04,445::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17df19a0] >Thread-338::DEBUG::2014-05-13 12:32:04,446::task::595::TaskManager.Task::(_updateState) Task=`86093a09-7c6c-4d34-8232-b15d64a727b7`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:32:04,446::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:32:04,446::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`cdca7b25-6818-42cc-9c47-9d30280c0261`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:32:04,447::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:32:04,447::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:32:04,447::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`cdca7b25-6818-42cc-9c47-9d30280c0261`::Granted request >Thread-338::DEBUG::2014-05-13 12:32:04,447::task::827::TaskManager.Task::(resourceAcquired) Task=`86093a09-7c6c-4d34-8232-b15d64a727b7`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:32:04,447::task::990::TaskManager.Task::(_decref) Task=`86093a09-7c6c-4d34-8232-b15d64a727b7`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:32:04,448::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:32:04,448::task::1185::TaskManager.Task::(prepare) Task=`86093a09-7c6c-4d34-8232-b15d64a727b7`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:32:04,449::task::595::TaskManager.Task::(_updateState) Task=`86093a09-7c6c-4d34-8232-b15d64a727b7`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:32:04,449::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:32:04,449::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:32:04,449::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:32:04,449::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:32:04,450::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:32:04,450::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:32:04,450::task::990::TaskManager.Task::(_decref) Task=`86093a09-7c6c-4d34-8232-b15d64a727b7`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 12:32:04,780::task::595::TaskManager.Task::(_updateState) Task=`bf5c0ee7-61b2-400d-9234-c6869dbb378f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:32:04,780::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:32:04,780::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000705137', 'lastCheck': '3.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:32:04,780::task::1185::TaskManager.Task::(prepare) Task=`bf5c0ee7-61b2-400d-9234-c6869dbb378f`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000705137', 'lastCheck': '3.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:32:04,780::task::595::TaskManager.Task::(_updateState) Task=`bf5c0ee7-61b2-400d-9234-c6869dbb378f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:32:04,781::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:32:04,781::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:32:04,781::task::990::TaskManager.Task::(_decref) Task=`bf5c0ee7-61b2-400d-9234-c6869dbb378f`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:32:05,346::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:32:06,158::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:32:06,234::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317662 s, 32.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:32:08,247::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:32:08,336::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0432755 s, 23.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:32:08,364::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:32:10,349::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:32:10,437::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0451403 s, 22.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:32:10,989::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:32:10,997::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000632274 s, 6.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:32:11,385::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:32:12,450::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:32:12,526::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321642 s, 31.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:32:14,404::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:32:14,470::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:32:14,471::task::595::TaskManager.Task::(_updateState) Task=`d9b55b07-8043-4086-99ea-25767ac8cf82`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:32:14,471::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:32:14,472::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:32:14,472::task::1185::TaskManager.Task::(prepare) Task=`d9b55b07-8043-4086-99ea-25767ac8cf82`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:32:14,472::task::595::TaskManager.Task::(_updateState) Task=`d9b55b07-8043-4086-99ea-25767ac8cf82`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:32:14,472::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:32:14,472::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:32:14,472::task::990::TaskManager.Task::(_decref) Task=`d9b55b07-8043-4086-99ea-25767ac8cf82`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:32:14,477::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:32:14,477::task::595::TaskManager.Task::(_updateState) Task=`4ab47a55-0e75-4648-875a-63c68eae28a3`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:32:14,478::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:32:14,478::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`48323a18-1d75-40cb-9efb-f693d96e3642`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:32:14,478::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:32:14,479::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:32:14,479::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`48323a18-1d75-40cb-9efb-f693d96e3642`::Granted request >Thread-338::DEBUG::2014-05-13 12:32:14,479::task::827::TaskManager.Task::(resourceAcquired) Task=`4ab47a55-0e75-4648-875a-63c68eae28a3`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:32:14,479::task::990::TaskManager.Task::(_decref) Task=`4ab47a55-0e75-4648-875a-63c68eae28a3`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:32:14,480::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:32:14,480::task::1185::TaskManager.Task::(prepare) Task=`4ab47a55-0e75-4648-875a-63c68eae28a3`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:32:14,480::task::595::TaskManager.Task::(_updateState) Task=`4ab47a55-0e75-4648-875a-63c68eae28a3`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:32:14,481::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:32:14,481::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:32:14,481::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:32:14,481::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:32:14,481::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:32:14,481::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:32:14,482::task::990::TaskManager.Task::(_decref) Task=`4ab47a55-0e75-4648-875a-63c68eae28a3`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:32:14,539::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:32:14,621::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0385806 s, 26.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:32:16,634::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:32:16,709::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317208 s, 32.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:32:17,423::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:32:18,722::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:32:18,804::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387651 s, 26.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:32:19,912::task::595::TaskManager.Task::(_updateState) Task=`f853b118-5a25-4dbc-a6be-00e1a8cdf6b1`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:32:19,912::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:32:19,913::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000632274', 'lastCheck': '8.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:32:19,913::task::1185::TaskManager.Task::(prepare) Task=`f853b118-5a25-4dbc-a6be-00e1a8cdf6b1`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000632274', 'lastCheck': '8.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:32:19,913::task::595::TaskManager.Task::(_updateState) Task=`f853b118-5a25-4dbc-a6be-00e1a8cdf6b1`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:32:19,913::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:32:19,913::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:32:19,913::task::990::TaskManager.Task::(_decref) Task=`f853b118-5a25-4dbc-a6be-00e1a8cdf6b1`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:32:20,441::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:32:20,817::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:32:20,892::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0333923 s, 30.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:32:21,004::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:32:21,012::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000817936 s, 5.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:32:22,905::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:32:23,273::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.325895 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:32:23,460::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:32:24,502::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:32:24,503::task::595::TaskManager.Task::(_updateState) Task=`3403af6b-a406-412b-952f-d968d5353657`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:32:24,503::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:32:24,503::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:32:24,503::task::1185::TaskManager.Task::(prepare) Task=`3403af6b-a406-412b-952f-d968d5353657`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:32:24,503::task::595::TaskManager.Task::(_updateState) Task=`3403af6b-a406-412b-952f-d968d5353657`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:32:24,504::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:32:24,504::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:32:24,504::task::990::TaskManager.Task::(_decref) Task=`3403af6b-a406-412b-952f-d968d5353657`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:32:24,508::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:32:24,509::task::595::TaskManager.Task::(_updateState) Task=`b7a0e62a-c9f9-494e-a34f-b7c1bf53cfd3`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:32:24,509::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:32:24,509::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b12fc027-67ed-4e34-a7da-d963c27f75f1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:32:24,509::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:32:24,510::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:32:24,510::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b12fc027-67ed-4e34-a7da-d963c27f75f1`::Granted request >Thread-338::DEBUG::2014-05-13 12:32:24,510::task::827::TaskManager.Task::(resourceAcquired) Task=`b7a0e62a-c9f9-494e-a34f-b7c1bf53cfd3`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:32:24,510::task::990::TaskManager.Task::(_decref) Task=`b7a0e62a-c9f9-494e-a34f-b7c1bf53cfd3`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:32:24,511::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:32:24,511::task::1185::TaskManager.Task::(prepare) Task=`b7a0e62a-c9f9-494e-a34f-b7c1bf53cfd3`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:32:24,511::task::595::TaskManager.Task::(_updateState) Task=`b7a0e62a-c9f9-494e-a34f-b7c1bf53cfd3`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:32:24,511::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:32:24,512::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:32:24,512::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:32:24,512::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:32:24,512::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:32:24,512::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:32:24,512::task::990::TaskManager.Task::(_decref) Task=`b7a0e62a-c9f9-494e-a34f-b7c1bf53cfd3`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:32:25,285::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:32:25,360::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319837 s, 32.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:32:26,478::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:32:27,373::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:32:27,448::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322403 s, 31.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:32:29,461::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:32:29,499::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:32:29,544::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0398109 s, 25.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:32:31,019::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:32:31,027::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000815699 s, 5.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:32:31,557::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:32:31,638::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0367279 s, 27.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:32:32,518::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:32:33,651::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:32:33,733::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0371157 s, 27.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:32:34,531::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [7a000b17] >Thread-13::DEBUG::2014-05-13 12:32:34,532::task::595::TaskManager.Task::(_updateState) Task=`f6725c35-05b0-4657-9567-fbbbb655ac0a`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:32:34,532::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:32:34,532::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:32:34,533::task::1185::TaskManager.Task::(prepare) Task=`f6725c35-05b0-4657-9567-fbbbb655ac0a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:32:34,533::task::595::TaskManager.Task::(_updateState) Task=`f6725c35-05b0-4657-9567-fbbbb655ac0a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:32:34,533::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:32:34,533::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:32:34,533::task::990::TaskManager.Task::(_decref) Task=`f6725c35-05b0-4657-9567-fbbbb655ac0a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:32:34,538::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [7a000b17] >Thread-338::DEBUG::2014-05-13 12:32:34,538::task::595::TaskManager.Task::(_updateState) Task=`c6683542-658b-463f-be2a-6db5efe2c59c`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:32:34,538::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:32:34,538::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`83b00bb8-bf4b-40dc-8712-b179d22a4d2d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:32:34,539::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:32:34,539::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:32:34,539::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`83b00bb8-bf4b-40dc-8712-b179d22a4d2d`::Granted request >Thread-338::DEBUG::2014-05-13 12:32:34,539::task::827::TaskManager.Task::(resourceAcquired) Task=`c6683542-658b-463f-be2a-6db5efe2c59c`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:32:34,540::task::990::TaskManager.Task::(_decref) Task=`c6683542-658b-463f-be2a-6db5efe2c59c`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:32:34,540::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:32:34,541::task::1185::TaskManager.Task::(prepare) Task=`c6683542-658b-463f-be2a-6db5efe2c59c`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:32:34,541::task::595::TaskManager.Task::(_updateState) Task=`c6683542-658b-463f-be2a-6db5efe2c59c`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:32:34,541::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:32:34,541::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:32:34,542::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:32:34,542::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:32:34,542::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:32:34,542::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:32:34,542::task::990::TaskManager.Task::(_decref) Task=`c6683542-658b-463f-be2a-6db5efe2c59c`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 12:32:35,046::task::595::TaskManager.Task::(_updateState) Task=`5d814374-0df1-43a3-804a-1bd3700bdbe2`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:32:35,046::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:32:35,046::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000815699', 'lastCheck': '4.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:32:35,046::task::1185::TaskManager.Task::(prepare) Task=`5d814374-0df1-43a3-804a-1bd3700bdbe2`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000815699', 'lastCheck': '4.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:32:35,046::task::595::TaskManager.Task::(_updateState) Task=`5d814374-0df1-43a3-804a-1bd3700bdbe2`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:32:35,047::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:32:35,047::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:32:35,047::task::990::TaskManager.Task::(_decref) Task=`5d814374-0df1-43a3-804a-1bd3700bdbe2`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:32:35,539::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:32:35,747::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:32:35,823::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0333291 s, 30.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:32:37,836::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:32:38,194::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.315935 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:32:38,566::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:32:40,207::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:32:40,282::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321694 s, 31.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:32:41,034::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:32:41,042::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000624567 s, 6.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:32:41,586::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:32:42,295::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:32:42,370::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315225 s, 32.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:32:44,383::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:32:44,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363697 s, 28.2 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:32:44,561::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:32:44,562::task::595::TaskManager.Task::(_updateState) Task=`94c5e293-0f6b-4b3e-849b-b17ade214b0c`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:32:44,562::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:32:44,562::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:32:44,562::task::1185::TaskManager.Task::(prepare) Task=`94c5e293-0f6b-4b3e-849b-b17ade214b0c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:32:44,563::task::595::TaskManager.Task::(_updateState) Task=`94c5e293-0f6b-4b3e-849b-b17ade214b0c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:32:44,563::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:32:44,563::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:32:44,563::task::990::TaskManager.Task::(_decref) Task=`94c5e293-0f6b-4b3e-849b-b17ade214b0c`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:32:44,567::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:32:44,568::task::595::TaskManager.Task::(_updateState) Task=`151bfba2-9119-452d-93e5-0dc7946f5c75`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:32:44,568::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:32:44,568::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8ce34c6e-6cab-4d7b-b2f0-24355e1a32cd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:32:44,569::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:32:44,569::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:32:44,569::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8ce34c6e-6cab-4d7b-b2f0-24355e1a32cd`::Granted request >Thread-338::DEBUG::2014-05-13 12:32:44,569::task::827::TaskManager.Task::(resourceAcquired) Task=`151bfba2-9119-452d-93e5-0dc7946f5c75`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:32:44,569::task::990::TaskManager.Task::(_decref) Task=`151bfba2-9119-452d-93e5-0dc7946f5c75`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:32:44,570::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:32:44,570::task::1185::TaskManager.Task::(prepare) Task=`151bfba2-9119-452d-93e5-0dc7946f5c75`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:32:44,571::task::595::TaskManager.Task::(_updateState) Task=`151bfba2-9119-452d-93e5-0dc7946f5c75`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:32:44,571::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:32:44,571::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:32:44,571::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:32:44,571::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:32:44,572::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:32:44,572::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:32:44,572::task::990::TaskManager.Task::(_decref) Task=`151bfba2-9119-452d-93e5-0dc7946f5c75`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:32:44,606::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:32:46,475::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:32:46,551::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031792 s, 32.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:32:47,630::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:32:48,564::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:32:48,660::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0444665 s, 23.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:32:50,176::task::595::TaskManager.Task::(_updateState) Task=`1c09d567-88ee-4fa8-b72d-0173cd78e83a`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:32:50,176::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:32:50,177::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000624567', 'lastCheck': '9.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:32:50,177::task::1185::TaskManager.Task::(prepare) Task=`1c09d567-88ee-4fa8-b72d-0173cd78e83a`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000624567', 'lastCheck': '9.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:32:50,177::task::595::TaskManager.Task::(_updateState) Task=`1c09d567-88ee-4fa8-b72d-0173cd78e83a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:32:50,177::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:32:50,177::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:32:50,178::task::990::TaskManager.Task::(_decref) Task=`1c09d567-88ee-4fa8-b72d-0173cd78e83a`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:32:50,674::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:32:50,674::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:32:50,755::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365892 s, 28.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:32:51,050::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:32:51,061::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000626152 s, 6.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:32:52,768::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:32:52,853::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0424533 s, 24.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:32:53,695::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:32:54,590::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:32:54,591::task::595::TaskManager.Task::(_updateState) Task=`df36444a-5029-4006-9f32-8487c56cd64e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:32:54,591::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:32:54,591::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:32:54,591::task::1185::TaskManager.Task::(prepare) Task=`df36444a-5029-4006-9f32-8487c56cd64e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:32:54,591::task::595::TaskManager.Task::(_updateState) Task=`df36444a-5029-4006-9f32-8487c56cd64e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:32:54,592::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:32:54,592::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:32:54,592::task::990::TaskManager.Task::(_decref) Task=`df36444a-5029-4006-9f32-8487c56cd64e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:32:54,866::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-338::DEBUG::2014-05-13 12:32:54,903::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:32:54,904::task::595::TaskManager.Task::(_updateState) Task=`2e0c0c2a-ca23-4c00-a482-8e6f2333ee5e`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:32:54,904::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:32:54,905::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`540d8c65-f3ea-4232-ae9c-d8f8398685d4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:32:54,905::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:32:54,905::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:32:54,906::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`540d8c65-f3ea-4232-ae9c-d8f8398685d4`::Granted request >Thread-338::DEBUG::2014-05-13 12:32:54,906::task::827::TaskManager.Task::(resourceAcquired) Task=`2e0c0c2a-ca23-4c00-a482-8e6f2333ee5e`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:32:54,906::task::990::TaskManager.Task::(_decref) Task=`2e0c0c2a-ca23-4c00-a482-8e6f2333ee5e`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:32:54,907::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:32:54,908::task::1185::TaskManager.Task::(prepare) Task=`2e0c0c2a-ca23-4c00-a482-8e6f2333ee5e`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:32:54,908::task::595::TaskManager.Task::(_updateState) Task=`2e0c0c2a-ca23-4c00-a482-8e6f2333ee5e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:32:54,908::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:32:54,909::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:32:54,909::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:32:54,909::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:32:54,909::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:32:54,910::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:32:54,910::task::990::TaskManager.Task::(_decref) Task=`2e0c0c2a-ca23-4c00-a482-8e6f2333ee5e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:32:54,948::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0402713 s, 25.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:32:56,714::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:32:56,961::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:32:57,035::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317899 s, 32.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:32:59,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:32:59,140::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0492161 s, 20.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:32:59,733::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:33:01,068::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:33:01,076::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000638073 s, 6.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:33:01,153::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:01,243::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0477551 s, 21.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:33:02,751::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:33:03,256::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:03,331::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313237 s, 32.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:33:04,928::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2ec76adc] >Thread-13::DEBUG::2014-05-13 12:33:04,929::task::595::TaskManager.Task::(_updateState) Task=`43e975f6-9d48-4cf9-8ca7-f68101a99233`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:33:04,929::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:33:04,929::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:33:04,929::task::1185::TaskManager.Task::(prepare) Task=`43e975f6-9d48-4cf9-8ca7-f68101a99233`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:33:04,930::task::595::TaskManager.Task::(_updateState) Task=`43e975f6-9d48-4cf9-8ca7-f68101a99233`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:33:04,930::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:33:04,930::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:33:04,930::task::990::TaskManager.Task::(_decref) Task=`43e975f6-9d48-4cf9-8ca7-f68101a99233`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:33:04,936::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2ec76adc] >Thread-338::DEBUG::2014-05-13 12:33:04,936::task::595::TaskManager.Task::(_updateState) Task=`80ffe13f-9a6b-47f2-a31a-836bc2f41be2`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:33:04,936::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:33:04,937::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`84781250-0f79-4fae-9640-a46b88781c49`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:33:04,937::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:33:04,937::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:33:04,937::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`84781250-0f79-4fae-9640-a46b88781c49`::Granted request >Thread-338::DEBUG::2014-05-13 12:33:04,939::task::827::TaskManager.Task::(resourceAcquired) Task=`80ffe13f-9a6b-47f2-a31a-836bc2f41be2`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:33:04,939::task::990::TaskManager.Task::(_decref) Task=`80ffe13f-9a6b-47f2-a31a-836bc2f41be2`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:33:04,939::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:33:04,940::task::1185::TaskManager.Task::(prepare) Task=`80ffe13f-9a6b-47f2-a31a-836bc2f41be2`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:33:04,940::task::595::TaskManager.Task::(_updateState) Task=`80ffe13f-9a6b-47f2-a31a-836bc2f41be2`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:33:04,940::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:33:04,940::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:33:04,940::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:33:04,941::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:33:04,941::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:33:04,941::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:33:04,941::task::990::TaskManager.Task::(_decref) Task=`80ffe13f-9a6b-47f2-a31a-836bc2f41be2`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 12:33:05,308::task::595::TaskManager.Task::(_updateState) Task=`95db1925-0b82-420c-a61e-272e8f307b94`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:33:05,309::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:33:05,309::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000638073', 'lastCheck': '4.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:33:05,309::task::1185::TaskManager.Task::(prepare) Task=`95db1925-0b82-420c-a61e-272e8f307b94`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000638073', 'lastCheck': '4.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:33:05,309::task::595::TaskManager.Task::(_updateState) Task=`95db1925-0b82-420c-a61e-272e8f307b94`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:33:05,309::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:33:05,309::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:33:05,310::task::990::TaskManager.Task::(_decref) Task=`95db1925-0b82-420c-a61e-272e8f307b94`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:33:05,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:05,423::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359255 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:33:05,773::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:33:07,436::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:07,509::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311842 s, 32.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:33:08,792::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:33:09,522::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:09,599::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319824 s, 32.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:33:11,084::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:33:11,091::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000685205 s, 6.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:33:11,613::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:11,694::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0394476 s, 26.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:33:11,810::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:33:13,706::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:13,788::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379549 s, 27.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:33:14,828::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:33:14,960::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:33:14,960::task::595::TaskManager.Task::(_updateState) Task=`fa75a2e4-eb2b-4561-a786-9426d073fb1e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:33:14,960::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:33:14,961::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:33:14,961::task::1185::TaskManager.Task::(prepare) Task=`fa75a2e4-eb2b-4561-a786-9426d073fb1e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:33:14,961::task::595::TaskManager.Task::(_updateState) Task=`fa75a2e4-eb2b-4561-a786-9426d073fb1e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:33:14,961::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:33:14,961::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:33:14,961::task::990::TaskManager.Task::(_decref) Task=`fa75a2e4-eb2b-4561-a786-9426d073fb1e`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:33:14,966::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:33:14,966::task::595::TaskManager.Task::(_updateState) Task=`deea7ff9-77ab-48b5-88ee-026fbcf8def2`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:33:14,967::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:33:14,967::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f3d9e63a-73c7-4a2f-8832-110b075b8e58`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:33:14,967::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:33:14,967::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:33:14,967::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f3d9e63a-73c7-4a2f-8832-110b075b8e58`::Granted request >Thread-338::DEBUG::2014-05-13 12:33:14,968::task::827::TaskManager.Task::(resourceAcquired) Task=`deea7ff9-77ab-48b5-88ee-026fbcf8def2`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:33:14,968::task::990::TaskManager.Task::(_decref) Task=`deea7ff9-77ab-48b5-88ee-026fbcf8def2`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:33:14,969::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:33:14,969::task::1185::TaskManager.Task::(prepare) Task=`deea7ff9-77ab-48b5-88ee-026fbcf8def2`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:33:14,969::task::595::TaskManager.Task::(_updateState) Task=`deea7ff9-77ab-48b5-88ee-026fbcf8def2`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:33:14,969::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:33:14,969::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:33:14,969::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:33:14,970::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:33:14,970::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:33:14,970::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:33:14,970::task::990::TaskManager.Task::(_decref) Task=`deea7ff9-77ab-48b5-88ee-026fbcf8def2`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:33:15,801::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:15,878::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.034699 s, 29.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:33:17,848::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:33:17,891::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:18,246::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.313679 s, 3.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:33:20,259::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:20,337::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356701 s, 28.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:33:20,448::task::595::TaskManager.Task::(_updateState) Task=`151273a9-5bcd-4ccc-8806-4f2332998daa`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:33:20,448::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:33:20,448::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000685205', 'lastCheck': '9.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:33:20,449::task::1185::TaskManager.Task::(prepare) Task=`151273a9-5bcd-4ccc-8806-4f2332998daa`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000685205', 'lastCheck': '9.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:33:20,449::task::595::TaskManager.Task::(_updateState) Task=`151273a9-5bcd-4ccc-8806-4f2332998daa`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:33:20,449::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:33:20,449::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:33:20,449::task::990::TaskManager.Task::(_decref) Task=`151273a9-5bcd-4ccc-8806-4f2332998daa`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:33:20,866::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:33:21,098::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:33:21,107::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00150346 s, 2.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:33:22,350::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:22,430::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356426 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:33:23,888::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:33:24,442::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:24,522::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383996 s, 26.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:33:24,988::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:33:24,988::task::595::TaskManager.Task::(_updateState) Task=`5f761f0b-da6e-4021-94ea-f6a72bff4971`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:33:24,989::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:33:24,989::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:33:24,989::task::1185::TaskManager.Task::(prepare) Task=`5f761f0b-da6e-4021-94ea-f6a72bff4971`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:33:24,990::task::595::TaskManager.Task::(_updateState) Task=`5f761f0b-da6e-4021-94ea-f6a72bff4971`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:33:24,990::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:33:24,990::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:33:24,990::task::990::TaskManager.Task::(_decref) Task=`5f761f0b-da6e-4021-94ea-f6a72bff4971`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:33:24,995::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:33:24,995::task::595::TaskManager.Task::(_updateState) Task=`e9b6eaa5-4375-4a63-bf52-ee496e5fdd2a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:33:24,995::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:33:24,996::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`639a7e38-4d1b-4c1c-ae9c-5feb1508320f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:33:24,996::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:33:24,996::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:33:24,996::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`639a7e38-4d1b-4c1c-ae9c-5feb1508320f`::Granted request >Thread-338::DEBUG::2014-05-13 12:33:24,997::task::827::TaskManager.Task::(resourceAcquired) Task=`e9b6eaa5-4375-4a63-bf52-ee496e5fdd2a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:33:24,997::task::990::TaskManager.Task::(_decref) Task=`e9b6eaa5-4375-4a63-bf52-ee496e5fdd2a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:33:24,998::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:33:24,998::task::1185::TaskManager.Task::(prepare) Task=`e9b6eaa5-4375-4a63-bf52-ee496e5fdd2a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:33:24,998::task::595::TaskManager.Task::(_updateState) Task=`e9b6eaa5-4375-4a63-bf52-ee496e5fdd2a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:33:24,998::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:33:24,998::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:33:24,999::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:33:24,999::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:33:24,999::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:33:24,999::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:33:24,999::task::990::TaskManager.Task::(_decref) Task=`e9b6eaa5-4375-4a63-bf52-ee496e5fdd2a`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:33:26,535::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:26,617::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374628 s, 27.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:33:26,906::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:33:28,630::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:28,706::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0337316 s, 30.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:33:29,923::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:33:30,719::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:30,807::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0441879 s, 23.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:33:31,152::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:33:31,160::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00080012 s, 5.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:33:32,820::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:32,895::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320965 s, 31.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:33:32,949::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:33:34,908::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:34,991::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379091 s, 27.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:33:35,019::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:33:35,020::task::595::TaskManager.Task::(_updateState) Task=`cdaab070-1d18-4b8a-961a-f8fb2296fdb3`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:33:35,020::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:33:35,020::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:33:35,020::task::1185::TaskManager.Task::(prepare) Task=`cdaab070-1d18-4b8a-961a-f8fb2296fdb3`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:33:35,021::task::595::TaskManager.Task::(_updateState) Task=`cdaab070-1d18-4b8a-961a-f8fb2296fdb3`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:33:35,021::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:33:35,021::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:33:35,021::task::990::TaskManager.Task::(_decref) Task=`cdaab070-1d18-4b8a-961a-f8fb2296fdb3`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:33:35,026::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:33:35,026::task::595::TaskManager.Task::(_updateState) Task=`00e81157-782a-42e9-a89d-e69c8a06a1ec`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:33:35,026::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:33:35,027::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`13a00576-3429-40ec-bb7d-00b52da7aabd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:33:35,027::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:33:35,027::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:33:35,028::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`13a00576-3429-40ec-bb7d-00b52da7aabd`::Granted request >Thread-338::DEBUG::2014-05-13 12:33:35,028::task::827::TaskManager.Task::(resourceAcquired) Task=`00e81157-782a-42e9-a89d-e69c8a06a1ec`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:33:35,028::task::990::TaskManager.Task::(_decref) Task=`00e81157-782a-42e9-a89d-e69c8a06a1ec`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:33:35,029::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:33:35,029::task::1185::TaskManager.Task::(prepare) Task=`00e81157-782a-42e9-a89d-e69c8a06a1ec`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:33:35,029::task::595::TaskManager.Task::(_updateState) Task=`00e81157-782a-42e9-a89d-e69c8a06a1ec`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:33:35,030::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:33:35,030::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:33:35,030::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:33:35,030::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:33:35,030::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:33:35,031::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:33:35,031::task::990::TaskManager.Task::(_decref) Task=`00e81157-782a-42e9-a89d-e69c8a06a1ec`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 12:33:35,581::task::595::TaskManager.Task::(_updateState) Task=`fccb5d2f-0aca-443f-98f9-17832727bc3e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:33:35,581::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:33:35,581::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00080012', 'lastCheck': '4.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:33:35,581::task::1185::TaskManager.Task::(prepare) Task=`fccb5d2f-0aca-443f-98f9-17832727bc3e`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00080012', 'lastCheck': '4.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:33:35,581::task::595::TaskManager.Task::(_updateState) Task=`fccb5d2f-0aca-443f-98f9-17832727bc3e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:33:35,582::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:33:35,582::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:33:35,582::task::990::TaskManager.Task::(_decref) Task=`fccb5d2f-0aca-443f-98f9-17832727bc3e`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:33:35,969::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:33:37,003::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:37,078::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315389 s, 32.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:33:38,988::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:33:39,092::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:39,172::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0372803 s, 27.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:33:41,167::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:33:41,175::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000670006 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:33:41,185::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:41,264::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362263 s, 28.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:33:42,006::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:33:43,277::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:43,640::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.321447 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:33:45,025::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:33:45,049::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:33:45,049::task::595::TaskManager.Task::(_updateState) Task=`2842e922-7516-4020-82df-21175a114d3e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:33:45,050::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:33:45,050::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:33:45,050::task::1185::TaskManager.Task::(prepare) Task=`2842e922-7516-4020-82df-21175a114d3e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:33:45,050::task::595::TaskManager.Task::(_updateState) Task=`2842e922-7516-4020-82df-21175a114d3e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:33:45,050::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:33:45,051::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:33:45,051::task::990::TaskManager.Task::(_decref) Task=`2842e922-7516-4020-82df-21175a114d3e`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:33:45,055::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:33:45,056::task::595::TaskManager.Task::(_updateState) Task=`d7e7147e-96a8-4d4d-8c0b-c239de0ca6fd`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:33:45,056::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:33:45,056::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`53c85447-d1cd-4008-bb76-5ff2b310543a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:33:45,056::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:33:45,057::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:33:45,057::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`53c85447-d1cd-4008-bb76-5ff2b310543a`::Granted request >Thread-338::DEBUG::2014-05-13 12:33:45,057::task::827::TaskManager.Task::(resourceAcquired) Task=`d7e7147e-96a8-4d4d-8c0b-c239de0ca6fd`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:33:45,057::task::990::TaskManager.Task::(_decref) Task=`d7e7147e-96a8-4d4d-8c0b-c239de0ca6fd`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:33:45,058::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:33:45,058::task::1185::TaskManager.Task::(prepare) Task=`d7e7147e-96a8-4d4d-8c0b-c239de0ca6fd`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:33:45,058::task::595::TaskManager.Task::(_updateState) Task=`d7e7147e-96a8-4d4d-8c0b-c239de0ca6fd`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:33:45,058::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:33:45,058::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:33:45,059::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:33:45,059::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:33:45,059::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:33:45,059::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:33:45,059::task::990::TaskManager.Task::(_decref) Task=`d7e7147e-96a8-4d4d-8c0b-c239de0ca6fd`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:33:45,653::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:45,729::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319077 s, 32.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:33:47,742::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:47,820::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355852 s, 28.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:33:48,045::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:33:49,833::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:49,913::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0369648 s, 27.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:33:50,713::task::595::TaskManager.Task::(_updateState) Task=`c68b274a-b70f-49fd-858c-cf81f43f981e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:33:50,713::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:33:50,713::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000670006', 'lastCheck': '9.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:33:50,713::task::1185::TaskManager.Task::(prepare) Task=`c68b274a-b70f-49fd-858c-cf81f43f981e`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000670006', 'lastCheck': '9.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:33:50,713::task::595::TaskManager.Task::(_updateState) Task=`c68b274a-b70f-49fd-858c-cf81f43f981e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:33:50,714::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:33:50,714::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:33:50,714::task::990::TaskManager.Task::(_decref) Task=`c68b274a-b70f-49fd-858c-cf81f43f981e`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:33:51,064::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:33:51,182::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:33:51,189::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000670508 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:33:51,926::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:52,008::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0389494 s, 26.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:33:54,021::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:33:54,084::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:33:54,106::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379667 s, 27.0 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:33:55,076::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:33:55,120::task::595::TaskManager.Task::(_updateState) Task=`66ef3554-477b-43c6-9d53-f95929bc23d2`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:33:55,120::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:33:55,121::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:33:55,121::task::1185::TaskManager.Task::(prepare) Task=`66ef3554-477b-43c6-9d53-f95929bc23d2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:33:55,121::task::595::TaskManager.Task::(_updateState) Task=`66ef3554-477b-43c6-9d53-f95929bc23d2`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:33:55,121::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:33:55,121::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:33:55,121::task::990::TaskManager.Task::(_decref) Task=`66ef3554-477b-43c6-9d53-f95929bc23d2`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:33:55,126::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:33:55,126::task::595::TaskManager.Task::(_updateState) Task=`7ed0bd96-7d02-4859-9249-c0d603750299`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:33:55,126::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:33:55,127::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`dc72f9d9-b28b-485c-93a6-8257dba11306`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:33:55,127::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:33:55,127::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:33:55,127::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`dc72f9d9-b28b-485c-93a6-8257dba11306`::Granted request >Thread-338::DEBUG::2014-05-13 12:33:55,128::task::827::TaskManager.Task::(resourceAcquired) Task=`7ed0bd96-7d02-4859-9249-c0d603750299`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:33:55,128::task::990::TaskManager.Task::(_decref) Task=`7ed0bd96-7d02-4859-9249-c0d603750299`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:33:55,129::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:33:55,129::task::1185::TaskManager.Task::(prepare) Task=`7ed0bd96-7d02-4859-9249-c0d603750299`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:33:55,129::task::595::TaskManager.Task::(_updateState) Task=`7ed0bd96-7d02-4859-9249-c0d603750299`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:33:55,129::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:33:55,129::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:33:55,130::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:33:55,130::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:33:55,130::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:33:55,130::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:33:55,130::task::990::TaskManager.Task::(_decref) Task=`7ed0bd96-7d02-4859-9249-c0d603750299`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:33:56,118::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:56,199::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0385475 s, 26.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:33:57,102::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:33:58,212::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:33:58,298::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0430353 s, 23.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:34:00,119::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:34:00,311::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:34:00,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318544 s, 32.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:34:01,197::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:34:01,205::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000705683 s, 5.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:34:02,400::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:34:02,482::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0386753 s, 26.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:34:03,138::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:34:04,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:34:04,576::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0369951 s, 27.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:34:05,148::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [7a000b17] >Thread-13::DEBUG::2014-05-13 12:34:05,156::task::595::TaskManager.Task::(_updateState) Task=`27897a33-650d-461e-8edd-ce4d1332eb11`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:34:05,156::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:34:05,157::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:34:05,157::task::1185::TaskManager.Task::(prepare) Task=`27897a33-650d-461e-8edd-ce4d1332eb11`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:34:05,157::task::595::TaskManager.Task::(_updateState) Task=`27897a33-650d-461e-8edd-ce4d1332eb11`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:34:05,157::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:34:05,157::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:34:05,157::task::990::TaskManager.Task::(_decref) Task=`27897a33-650d-461e-8edd-ce4d1332eb11`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:34:05,163::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [7a000b17] >Thread-338::DEBUG::2014-05-13 12:34:05,163::task::595::TaskManager.Task::(_updateState) Task=`357c2f66-093a-4983-b343-cbbd84240026`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:34:05,163::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:34:05,164::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c2858716-aa45-4d08-bafd-010e439e3db2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:34:05,164::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:34:05,164::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:34:05,164::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c2858716-aa45-4d08-bafd-010e439e3db2`::Granted request >Thread-338::DEBUG::2014-05-13 12:34:05,165::task::827::TaskManager.Task::(resourceAcquired) Task=`357c2f66-093a-4983-b343-cbbd84240026`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:34:05,165::task::990::TaskManager.Task::(_decref) Task=`357c2f66-093a-4983-b343-cbbd84240026`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:34:05,166::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:34:05,166::task::1185::TaskManager.Task::(prepare) Task=`357c2f66-093a-4983-b343-cbbd84240026`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:34:05,166::task::595::TaskManager.Task::(_updateState) Task=`357c2f66-093a-4983-b343-cbbd84240026`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:34:05,166::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:34:05,166::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:34:05,167::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:34:05,167::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:34:05,167::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:34:05,167::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:34:05,167::task::990::TaskManager.Task::(_decref) Task=`357c2f66-093a-4983-b343-cbbd84240026`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 12:34:05,841::task::595::TaskManager.Task::(_updateState) Task=`f2a74e71-6266-4f39-a653-18d410cf216a`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:34:05,842::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:34:05,842::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000705683', 'lastCheck': '4.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:34:05,842::task::1185::TaskManager.Task::(prepare) Task=`f2a74e71-6266-4f39-a653-18d410cf216a`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000705683', 'lastCheck': '4.6', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:34:05,842::task::595::TaskManager.Task::(_updateState) Task=`f2a74e71-6266-4f39-a653-18d410cf216a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:34:05,842::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:34:05,843::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:34:05,843::task::990::TaskManager.Task::(_decref) Task=`f2a74e71-6266-4f39-a653-18d410cf216a`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:34:06,155::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:34:06,589::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:34:06,665::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317549 s, 32.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:34:08,677::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:34:08,757::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0366308 s, 28.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:34:09,173::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:34:10,770::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:34:10,865::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0521713 s, 19.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:34:11,212::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:34:11,220::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00059437 s, 6.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:34:12,193::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:34:12,878::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:34:12,954::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318049 s, 32.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:34:14,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:34:15,043::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323482 s, 31.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:34:15,186::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:34:15,186::task::595::TaskManager.Task::(_updateState) Task=`95c91b8f-fa8c-4378-bebf-ff5b8fd0d584`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:34:15,187::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:34:15,187::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:34:15,187::task::1185::TaskManager.Task::(prepare) Task=`95c91b8f-fa8c-4378-bebf-ff5b8fd0d584`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:34:15,187::task::595::TaskManager.Task::(_updateState) Task=`95c91b8f-fa8c-4378-bebf-ff5b8fd0d584`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:34:15,188::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:34:15,188::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:34:15,188::task::990::TaskManager.Task::(_decref) Task=`95c91b8f-fa8c-4378-bebf-ff5b8fd0d584`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:34:15,193::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:34:15,193::task::595::TaskManager.Task::(_updateState) Task=`bcc97cd6-2a8f-4217-b463-6205c0c2cb0b`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:34:15,193::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:34:15,193::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3f5790e6-3f43-4424-9e5e-84a46d0dc3a8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:34:15,194::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:34:15,194::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:34:15,194::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3f5790e6-3f43-4424-9e5e-84a46d0dc3a8`::Granted request >Thread-338::DEBUG::2014-05-13 12:34:15,196::task::827::TaskManager.Task::(resourceAcquired) Task=`bcc97cd6-2a8f-4217-b463-6205c0c2cb0b`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:34:15,196::task::990::TaskManager.Task::(_decref) Task=`bcc97cd6-2a8f-4217-b463-6205c0c2cb0b`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:34:15,197::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:34:15,197::task::1185::TaskManager.Task::(prepare) Task=`bcc97cd6-2a8f-4217-b463-6205c0c2cb0b`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:34:15,197::task::595::TaskManager.Task::(_updateState) Task=`bcc97cd6-2a8f-4217-b463-6205c0c2cb0b`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:34:15,197::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:34:15,198::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:34:15,198::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:34:15,198::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:34:15,198::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:34:15,198::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:34:15,199::task::990::TaskManager.Task::(_decref) Task=`bcc97cd6-2a8f-4217-b463-6205c0c2cb0b`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:34:15,233::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:34:17,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:34:17,142::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0427794 s, 23.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:34:18,257::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:34:19,155::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:34:19,234::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0372048 s, 27.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:34:20,973::task::595::TaskManager.Task::(_updateState) Task=`0e96a7b1-180a-4065-ae87-c386059cfc26`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:34:20,973::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:34:20,973::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00059437', 'lastCheck': '9.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:34:20,974::task::1185::TaskManager.Task::(prepare) Task=`0e96a7b1-180a-4065-ae87-c386059cfc26`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00059437', 'lastCheck': '9.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:34:20,974::task::595::TaskManager.Task::(_updateState) Task=`0e96a7b1-180a-4065-ae87-c386059cfc26`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:34:20,974::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:34:20,974::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:34:20,974::task::990::TaskManager.Task::(_decref) Task=`0e96a7b1-180a-4065-ae87-c386059cfc26`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 12:34:21,227::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:34:21,234::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000671377 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:34:21,247::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:34:21,275::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:34:21,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0541315 s, 18.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:34:23,357::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:34:23,731::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.332611 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:34:24,294::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:34:25,217::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [19876d15] >Thread-13::DEBUG::2014-05-13 12:34:25,217::task::595::TaskManager.Task::(_updateState) Task=`b5431691-5eb5-43f9-8734-5eaeb7929944`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:34:25,217::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:34:25,218::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:34:25,218::task::1185::TaskManager.Task::(prepare) Task=`b5431691-5eb5-43f9-8734-5eaeb7929944`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:34:25,218::task::595::TaskManager.Task::(_updateState) Task=`b5431691-5eb5-43f9-8734-5eaeb7929944`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:34:25,218::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:34:25,218::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:34:25,218::task::990::TaskManager.Task::(_decref) Task=`b5431691-5eb5-43f9-8734-5eaeb7929944`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:34:25,223::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [19876d15] >Thread-338::DEBUG::2014-05-13 12:34:25,224::task::595::TaskManager.Task::(_updateState) Task=`67b80f34-3824-4122-9981-0f736d804564`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:34:25,224::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:34:25,224::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1eccfbab-d76c-4e3d-9a79-51e8d7f208b3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:34:25,224::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:34:25,225::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:34:25,225::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1eccfbab-d76c-4e3d-9a79-51e8d7f208b3`::Granted request >Thread-338::DEBUG::2014-05-13 12:34:25,225::task::827::TaskManager.Task::(resourceAcquired) Task=`67b80f34-3824-4122-9981-0f736d804564`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:34:25,225::task::990::TaskManager.Task::(_decref) Task=`67b80f34-3824-4122-9981-0f736d804564`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:34:25,226::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:34:25,226::task::1185::TaskManager.Task::(prepare) Task=`67b80f34-3824-4122-9981-0f736d804564`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:34:25,226::task::595::TaskManager.Task::(_updateState) Task=`67b80f34-3824-4122-9981-0f736d804564`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:34:25,227::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:34:25,227::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:34:25,227::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:34:25,227::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:34:25,227::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:34:25,228::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:34:25,228::task::990::TaskManager.Task::(_decref) Task=`67b80f34-3824-4122-9981-0f736d804564`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:34:25,744::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:34:25,930::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.142937 s, 7.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:34:27,314::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:34:27,943::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:34:28,018::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310337 s, 33.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:34:30,031::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:34:30,108::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309592 s, 33.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:34:30,334::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:34:31,242::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:34:31,250::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00070271 s, 5.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:34:32,121::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:34:32,202::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387412 s, 26.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:34:33,351::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:34:34,215::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:34:34,295::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387726 s, 26.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:34:35,247::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:34:35,253::task::595::TaskManager.Task::(_updateState) Task=`36aecd98-1001-477e-a693-c7c07eaf137e`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:34:35,253::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:34:35,253::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:34:35,253::task::1185::TaskManager.Task::(prepare) Task=`36aecd98-1001-477e-a693-c7c07eaf137e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:34:35,254::task::595::TaskManager.Task::(_updateState) Task=`36aecd98-1001-477e-a693-c7c07eaf137e`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:34:35,254::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:34:35,254::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:34:35,254::task::990::TaskManager.Task::(_decref) Task=`36aecd98-1001-477e-a693-c7c07eaf137e`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:34:35,260::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:34:35,260::task::595::TaskManager.Task::(_updateState) Task=`b4cf42f5-38a9-499c-83ff-ca95207e823b`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:34:35,261::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:34:35,261::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5b2f787a-a300-46f5-b85a-5a06f844bdab`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:34:35,261::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:34:35,261::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:34:35,262::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5b2f787a-a300-46f5-b85a-5a06f844bdab`::Granted request >Thread-338::DEBUG::2014-05-13 12:34:35,262::task::827::TaskManager.Task::(resourceAcquired) Task=`b4cf42f5-38a9-499c-83ff-ca95207e823b`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:34:35,262::task::990::TaskManager.Task::(_decref) Task=`b4cf42f5-38a9-499c-83ff-ca95207e823b`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:34:35,263::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:34:35,263::task::1185::TaskManager.Task::(prepare) Task=`b4cf42f5-38a9-499c-83ff-ca95207e823b`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:34:35,263::task::595::TaskManager.Task::(_updateState) Task=`b4cf42f5-38a9-499c-83ff-ca95207e823b`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:34:35,263::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:34:35,264::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:34:35,264::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:34:35,264::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:34:35,264::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:34:35,264::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:34:35,265::task::990::TaskManager.Task::(_decref) Task=`b4cf42f5-38a9-499c-83ff-ca95207e823b`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 12:34:36,112::task::595::TaskManager.Task::(_updateState) Task=`e9cbef36-646c-4c92-afee-f08e99a0cbef`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:34:36,112::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:34:36,113::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00070271', 'lastCheck': '4.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:34:36,113::task::1185::TaskManager.Task::(prepare) Task=`e9cbef36-646c-4c92-afee-f08e99a0cbef`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00070271', 'lastCheck': '4.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:34:36,113::task::595::TaskManager.Task::(_updateState) Task=`e9cbef36-646c-4c92-afee-f08e99a0cbef`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:34:36,113::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:34:36,113::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:34:36,113::task::990::TaskManager.Task::(_decref) Task=`e9cbef36-646c-4c92-afee-f08e99a0cbef`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:34:36,308::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:34:36,371::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:34:36,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319188 s, 32.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:34:38,399::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:34:38,493::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0513312 s, 19.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:34:39,390::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:34:40,506::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:34:40,589::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0375178 s, 27.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:34:41,257::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:34:41,265::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000808267 s, 5.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:34:42,409::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:34:42,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:34:42,678::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316559 s, 32.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:34:44,691::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:34:44,773::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374015 s, 27.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:34:45,283::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [303c64a0] >Thread-13::DEBUG::2014-05-13 12:34:45,283::task::595::TaskManager.Task::(_updateState) Task=`30ba1708-4594-4561-95d9-a4fc1e0a8a95`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:34:45,284::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:34:45,284::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:34:45,284::task::1185::TaskManager.Task::(prepare) Task=`30ba1708-4594-4561-95d9-a4fc1e0a8a95`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:34:45,284::task::595::TaskManager.Task::(_updateState) Task=`30ba1708-4594-4561-95d9-a4fc1e0a8a95`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:34:45,284::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:34:45,285::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:34:45,285::task::990::TaskManager.Task::(_decref) Task=`30ba1708-4594-4561-95d9-a4fc1e0a8a95`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:34:45,289::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [303c64a0] >Thread-338::DEBUG::2014-05-13 12:34:45,289::task::595::TaskManager.Task::(_updateState) Task=`947d5958-b26a-4621-b66c-600fafb9cbd3`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:34:45,290::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:34:45,290::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fafe6e19-d2ea-4eae-b8e8-e9e2107a7aa0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:34:45,290::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:34:45,290::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:34:45,290::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fafe6e19-d2ea-4eae-b8e8-e9e2107a7aa0`::Granted request >Thread-338::DEBUG::2014-05-13 12:34:45,291::task::827::TaskManager.Task::(resourceAcquired) Task=`947d5958-b26a-4621-b66c-600fafb9cbd3`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:34:45,291::task::990::TaskManager.Task::(_decref) Task=`947d5958-b26a-4621-b66c-600fafb9cbd3`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:34:45,292::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:34:45,292::task::1185::TaskManager.Task::(prepare) Task=`947d5958-b26a-4621-b66c-600fafb9cbd3`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:34:45,292::task::595::TaskManager.Task::(_updateState) Task=`947d5958-b26a-4621-b66c-600fafb9cbd3`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:34:45,292::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:34:45,292::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:34:45,292::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:34:45,293::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:34:45,293::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:34:45,293::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:34:45,293::task::990::TaskManager.Task::(_decref) Task=`947d5958-b26a-4621-b66c-600fafb9cbd3`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:34:45,430::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:34:46,786::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:34:46,861::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320057 s, 32.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:34:48,449::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:34:48,874::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:34:48,950::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323364 s, 31.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:34:50,963::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-13::DEBUG::2014-05-13 12:34:51,250::task::595::TaskManager.Task::(_updateState) Task=`214f80c3-b5d2-4f2e-97bd-cf617b9fa2c4`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:34:51,250::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:34:51,250::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000808267', 'lastCheck': '10.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:34:51,250::task::1185::TaskManager.Task::(prepare) Task=`214f80c3-b5d2-4f2e-97bd-cf617b9fa2c4`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000808267', 'lastCheck': '10.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:34:51,250::task::595::TaskManager.Task::(_updateState) Task=`214f80c3-b5d2-4f2e-97bd-cf617b9fa2c4`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:34:51,251::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:34:51,251::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:34:51,251::task::990::TaskManager.Task::(_decref) Task=`214f80c3-b5d2-4f2e-97bd-cf617b9fa2c4`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 12:34:51,275::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:34:51,290::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00066881 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:34:51,365::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.360335 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:34:51,468::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:34:53,389::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:34:53,466::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357006 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:34:54,487::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:34:55,310::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4dbe430a] >Thread-13::DEBUG::2014-05-13 12:34:55,311::task::595::TaskManager.Task::(_updateState) Task=`91e8cdb3-820a-4116-ad57-32148f11b7da`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:34:55,311::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:34:55,311::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:34:55,312::task::1185::TaskManager.Task::(prepare) Task=`91e8cdb3-820a-4116-ad57-32148f11b7da`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:34:55,312::task::595::TaskManager.Task::(_updateState) Task=`91e8cdb3-820a-4116-ad57-32148f11b7da`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:34:55,312::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:34:55,312::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:34:55,312::task::990::TaskManager.Task::(_decref) Task=`91e8cdb3-820a-4116-ad57-32148f11b7da`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:34:55,317::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4dbe430a] >Thread-338::DEBUG::2014-05-13 12:34:55,318::task::595::TaskManager.Task::(_updateState) Task=`2e969163-f298-4379-a2d9-cb826a6c5221`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:34:55,318::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:34:55,318::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`02c45e45-ce62-44c7-b684-07cf6a9ef5a3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:34:55,319::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:34:55,319::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:34:55,319::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`02c45e45-ce62-44c7-b684-07cf6a9ef5a3`::Granted request >Thread-338::DEBUG::2014-05-13 12:34:55,319::task::827::TaskManager.Task::(resourceAcquired) Task=`2e969163-f298-4379-a2d9-cb826a6c5221`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:34:55,319::task::990::TaskManager.Task::(_decref) Task=`2e969163-f298-4379-a2d9-cb826a6c5221`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:34:55,320::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:34:55,320::task::1185::TaskManager.Task::(prepare) Task=`2e969163-f298-4379-a2d9-cb826a6c5221`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:34:55,321::task::595::TaskManager.Task::(_updateState) Task=`2e969163-f298-4379-a2d9-cb826a6c5221`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:34:55,321::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:34:55,321::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:34:55,321::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:34:55,321::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:34:55,322::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:34:55,322::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:34:55,322::task::990::TaskManager.Task::(_decref) Task=`2e969163-f298-4379-a2d9-cb826a6c5221`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:34:55,480::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:34:55,575::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0461881 s, 22.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:34:57,508::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:34:57,588::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:34:57,664::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0328756 s, 31.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:34:59,677::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:34:59,758::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0376514 s, 27.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:35:00,528::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:35:01,312::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:35:01,320::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000740155 s, 5.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:35:01,771::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:01,852::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387659 s, 26.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:35:03,548::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:35:03,865::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:03,954::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.045157 s, 22.7 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:35:05,341::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:35:05,349::task::595::TaskManager.Task::(_updateState) Task=`3694aa73-0042-4de1-bccd-40923dc1dbcf`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:35:05,349::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:35:05,350::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:35:05,350::task::1185::TaskManager.Task::(prepare) Task=`3694aa73-0042-4de1-bccd-40923dc1dbcf`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:35:05,350::task::595::TaskManager.Task::(_updateState) Task=`3694aa73-0042-4de1-bccd-40923dc1dbcf`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:35:05,350::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:35:05,351::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:35:05,351::task::990::TaskManager.Task::(_decref) Task=`3694aa73-0042-4de1-bccd-40923dc1dbcf`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:35:05,355::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:35:05,355::task::595::TaskManager.Task::(_updateState) Task=`b10fa6f1-eb65-452c-8f9a-991047f69164`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:35:05,356::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:35:05,356::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`90e1cf4f-0831-4039-920d-6daa7deef826`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:35:05,356::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:35:05,357::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:35:05,357::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`90e1cf4f-0831-4039-920d-6daa7deef826`::Granted request >Thread-338::DEBUG::2014-05-13 12:35:05,357::task::827::TaskManager.Task::(resourceAcquired) Task=`b10fa6f1-eb65-452c-8f9a-991047f69164`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:35:05,357::task::990::TaskManager.Task::(_decref) Task=`b10fa6f1-eb65-452c-8f9a-991047f69164`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:35:05,358::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:35:05,358::task::1185::TaskManager.Task::(prepare) Task=`b10fa6f1-eb65-452c-8f9a-991047f69164`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:35:05,358::task::595::TaskManager.Task::(_updateState) Task=`b10fa6f1-eb65-452c-8f9a-991047f69164`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:35:05,359::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:35:05,359::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:35:05,359::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:35:05,359::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:35:05,359::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:35:05,359::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:35:05,360::task::990::TaskManager.Task::(_decref) Task=`b10fa6f1-eb65-452c-8f9a-991047f69164`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:35:05,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:06,057::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0477735 s, 21.4 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:35:06,384::task::595::TaskManager.Task::(_updateState) Task=`63200260-ccf5-4cb7-8363-9df420f50991`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:35:06,384::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:35:06,385::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000740155', 'lastCheck': '5.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:35:06,385::task::1185::TaskManager.Task::(prepare) Task=`63200260-ccf5-4cb7-8363-9df420f50991`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000740155', 'lastCheck': '5.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:35:06,385::task::595::TaskManager.Task::(_updateState) Task=`63200260-ccf5-4cb7-8363-9df420f50991`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:35:06,385::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:35:06,385::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:35:06,385::task::990::TaskManager.Task::(_decref) Task=`63200260-ccf5-4cb7-8363-9df420f50991`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:35:06,567::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:35:08,070::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:08,149::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359089 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:35:09,585::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:35:10,162::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:10,242::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362353 s, 28.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:35:11,325::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 41830dad-f012-4b4d-b636-863f66d5ca4b >Thread-322::DEBUG::2014-05-13 12:35:11,325::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-322::DEBUG::2014-05-13 12:35:11,328::blockSD::336::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:35:11,336::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.00132629 s, 1.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:35:11,337::misc::287::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.00132629 s, 1.5 MB/s'], size: 2048 >Thread-322::DEBUG::2014-05-13 12:35:11,337::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-322::WARNING::2014-05-13 12:35:11,337::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-322::DEBUG::2014-05-13 12:35:11,337::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-322::DEBUG::2014-05-13 12:35:11,338::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:35:11,338::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:35:11,338::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:35:11,338::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:35:11,339::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:35:11,340::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 12:35:11,558::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:35:11,559::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:35:11,560::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'LOCKPOLICY=', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', 'DESCRIPTION=ISCSI', 'LOGBLKSIZE=512', 'TYPE=ISCSI', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'PHYBLKSIZE=512', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', 'CLASS=Data', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'POOL_DESCRIPTION=DC_2', 'POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'MASTER_VERSION=1', 'ROLE=Master', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', '_SHA_CKSUM=28bb2d7cc60e0d779eb69fe93d050c16c769f069', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0'] >Thread-322::INFO::2014-05-13 12:35:11,561::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_imageNS already registered >Thread-322::INFO::2014-05-13 12:35:11,561::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_volumeNS already registered >Thread-322::INFO::2014-05-13 12:35:11,561::blockSD::452::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_lvmActivationNS already registered >Thread-322::DEBUG::2014-05-13 12:35:11,562::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 12:35:11,731::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:35:11,735::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:35:11,743::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00071069 s, 5.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:35:12,255::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:12,330::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316053 s, 32.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:35:12,604::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:35:14,343::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:14,423::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0371128 s, 27.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:35:15,379::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:35:15,380::task::595::TaskManager.Task::(_updateState) Task=`7a89454d-4044-4a9d-b829-b589654ad788`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:35:15,380::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:35:15,380::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:35:15,380::task::1185::TaskManager.Task::(prepare) Task=`7a89454d-4044-4a9d-b829-b589654ad788`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:35:15,380::task::595::TaskManager.Task::(_updateState) Task=`7a89454d-4044-4a9d-b829-b589654ad788`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:35:15,381::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:35:15,381::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:35:15,381::task::990::TaskManager.Task::(_decref) Task=`7a89454d-4044-4a9d-b829-b589654ad788`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:35:15,385::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:35:15,386::task::595::TaskManager.Task::(_updateState) Task=`40a0dd4c-5141-4daa-ad98-5b889a0c1b1a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:35:15,386::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:35:15,386::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`da8cf388-dccc-44dc-a7ae-85262bcf6c61`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:35:15,386::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:35:15,387::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:35:15,387::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`da8cf388-dccc-44dc-a7ae-85262bcf6c61`::Granted request >Thread-338::DEBUG::2014-05-13 12:35:15,387::task::827::TaskManager.Task::(resourceAcquired) Task=`40a0dd4c-5141-4daa-ad98-5b889a0c1b1a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:35:15,387::task::990::TaskManager.Task::(_decref) Task=`40a0dd4c-5141-4daa-ad98-5b889a0c1b1a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:35:15,388::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:35:15,388::task::1185::TaskManager.Task::(prepare) Task=`40a0dd4c-5141-4daa-ad98-5b889a0c1b1a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:35:15,388::task::595::TaskManager.Task::(_updateState) Task=`40a0dd4c-5141-4daa-ad98-5b889a0c1b1a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:35:15,388::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:35:15,389::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:35:15,389::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:35:15,389::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:35:15,389::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:35:15,389::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:35:15,390::task::990::TaskManager.Task::(_decref) Task=`40a0dd4c-5141-4daa-ad98-5b889a0c1b1a`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:35:15,623::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:35:16,436::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:16,512::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320179 s, 32.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:35:18,525::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:18,614::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0459842 s, 22.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:35:18,642::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:35:20,627::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:20,706::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0367271 s, 27.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:35:21,522::task::595::TaskManager.Task::(_updateState) Task=`2b239d24-d4ad-4cf9-8de8-0d3f718d11ad`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:35:21,565::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:35:21,565::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00071069', 'lastCheck': '9.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:35:21,565::task::1185::TaskManager.Task::(prepare) Task=`2b239d24-d4ad-4cf9-8de8-0d3f718d11ad`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00071069', 'lastCheck': '9.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:35:21,566::task::595::TaskManager.Task::(_updateState) Task=`2b239d24-d4ad-4cf9-8de8-0d3f718d11ad`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:35:21,566::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:35:21,566::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:35:21,566::task::990::TaskManager.Task::(_decref) Task=`2b239d24-d4ad-4cf9-8de8-0d3f718d11ad`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:35:21,659::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:35:21,750::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:35:21,758::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000722082 s, 5.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:35:22,719::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:22,800::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382767 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:35:24,677::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:35:24,814::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:24,893::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0371815 s, 27.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:35:25,411::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5fad7764] >Thread-13::DEBUG::2014-05-13 12:35:25,412::task::595::TaskManager.Task::(_updateState) Task=`26fa91e4-b1ca-4135-9f91-0bfca4ddb02c`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:35:25,412::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:35:25,413::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:35:25,413::task::1185::TaskManager.Task::(prepare) Task=`26fa91e4-b1ca-4135-9f91-0bfca4ddb02c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:35:25,413::task::595::TaskManager.Task::(_updateState) Task=`26fa91e4-b1ca-4135-9f91-0bfca4ddb02c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:35:25,413::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:35:25,413::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:35:25,413::task::990::TaskManager.Task::(_decref) Task=`26fa91e4-b1ca-4135-9f91-0bfca4ddb02c`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:35:25,418::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5fad7764] >Thread-338::DEBUG::2014-05-13 12:35:25,419::task::595::TaskManager.Task::(_updateState) Task=`6c8dd112-8277-4ed6-a248-0d43c011eed2`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:35:25,419::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:35:25,419::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`dbd9262f-128e-4184-b167-2120738df1ee`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:35:25,419::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:35:25,420::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:35:25,420::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`dbd9262f-128e-4184-b167-2120738df1ee`::Granted request >Thread-338::DEBUG::2014-05-13 12:35:25,420::task::827::TaskManager.Task::(resourceAcquired) Task=`6c8dd112-8277-4ed6-a248-0d43c011eed2`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:35:25,420::task::990::TaskManager.Task::(_decref) Task=`6c8dd112-8277-4ed6-a248-0d43c011eed2`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:35:25,421::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:35:25,421::task::1185::TaskManager.Task::(prepare) Task=`6c8dd112-8277-4ed6-a248-0d43c011eed2`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:35:25,422::task::595::TaskManager.Task::(_updateState) Task=`6c8dd112-8277-4ed6-a248-0d43c011eed2`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:35:25,422::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:35:25,422::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:35:25,422::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:35:25,422::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:35:25,423::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:35:25,423::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:35:25,423::task::990::TaskManager.Task::(_decref) Task=`6c8dd112-8277-4ed6-a248-0d43c011eed2`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:35:26,906::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:26,981::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035675 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:35:27,702::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:35:28,989::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:29,063::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0308136 s, 33.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:35:30,722::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:35:31,076::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:31,163::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0448694 s, 22.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:35:31,765::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:35:31,773::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000831724 s, 4.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:35:33,177::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:33,254::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0328771 s, 31.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:35:33,739::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:35:35,267::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:35,347::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036639 s, 27.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:35:35,442::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:35:35,443::task::595::TaskManager.Task::(_updateState) Task=`42ede0d7-e6d7-4c4a-9e50-24e12ed58541`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:35:35,443::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:35:35,443::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:35:35,443::task::1185::TaskManager.Task::(prepare) Task=`42ede0d7-e6d7-4c4a-9e50-24e12ed58541`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:35:35,444::task::595::TaskManager.Task::(_updateState) Task=`42ede0d7-e6d7-4c4a-9e50-24e12ed58541`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:35:35,444::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:35:35,444::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:35:35,444::task::990::TaskManager.Task::(_decref) Task=`42ede0d7-e6d7-4c4a-9e50-24e12ed58541`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:35:35,449::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:35:35,449::task::595::TaskManager.Task::(_updateState) Task=`861a7dd1-6705-40a7-a5fc-f5deae251d6e`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:35:35,449::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:35:35,450::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d818d4b8-a14c-4f13-9124-591e10e0257e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:35:35,450::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:35:35,450::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:35:35,451::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d818d4b8-a14c-4f13-9124-591e10e0257e`::Granted request >Thread-338::DEBUG::2014-05-13 12:35:35,451::task::827::TaskManager.Task::(resourceAcquired) Task=`861a7dd1-6705-40a7-a5fc-f5deae251d6e`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:35:35,451::task::990::TaskManager.Task::(_decref) Task=`861a7dd1-6705-40a7-a5fc-f5deae251d6e`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:35:35,452::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:35:35,452::task::1185::TaskManager.Task::(prepare) Task=`861a7dd1-6705-40a7-a5fc-f5deae251d6e`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:35:35,452::task::595::TaskManager.Task::(_updateState) Task=`861a7dd1-6705-40a7-a5fc-f5deae251d6e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:35:35,452::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:35:35,453::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:35:35,453::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:35:35,453::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:35:35,453::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:35:35,453::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:35:35,454::task::990::TaskManager.Task::(_decref) Task=`861a7dd1-6705-40a7-a5fc-f5deae251d6e`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 12:35:36,701::task::595::TaskManager.Task::(_updateState) Task=`341e0d3e-114a-4a2b-96d8-ff0283040e90`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:35:36,702::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:35:36,702::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000831724', 'lastCheck': '4.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:35:36,702::task::1185::TaskManager.Task::(prepare) Task=`341e0d3e-114a-4a2b-96d8-ff0283040e90`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000831724', 'lastCheck': '4.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:35:36,702::task::595::TaskManager.Task::(_updateState) Task=`341e0d3e-114a-4a2b-96d8-ff0283040e90`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:35:36,702::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:35:36,702::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:35:36,703::task::990::TaskManager.Task::(_decref) Task=`341e0d3e-114a-4a2b-96d8-ff0283040e90`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:35:36,758::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:35:37,360::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:37,436::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.032312 s, 31.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:35:39,449::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:39,548::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0370219 s, 27.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:35:39,779::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:35:41,561::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:41,644::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0392366 s, 26.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:35:41,781::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:35:41,789::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000638049 s, 6.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:35:42,798::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:35:43,657::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:44,061::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.362412 s, 2.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:35:45,470::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:35:45,471::task::595::TaskManager.Task::(_updateState) Task=`71cb2a72-30d8-4e81-8048-5e8b2c2b3612`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:35:45,471::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:35:45,471::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:35:45,472::task::1185::TaskManager.Task::(prepare) Task=`71cb2a72-30d8-4e81-8048-5e8b2c2b3612`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:35:45,472::task::595::TaskManager.Task::(_updateState) Task=`71cb2a72-30d8-4e81-8048-5e8b2c2b3612`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:35:45,472::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:35:45,472::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:35:45,472::task::990::TaskManager.Task::(_decref) Task=`71cb2a72-30d8-4e81-8048-5e8b2c2b3612`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:35:45,477::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:35:45,477::task::595::TaskManager.Task::(_updateState) Task=`ff3fca80-23af-4fe3-b6ec-b00ca27eda25`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:35:45,477::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:35:45,478::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fd2b08a9-9569-470e-9ba2-b774d159bbe5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:35:45,478::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:35:45,478::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:35:45,478::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fd2b08a9-9569-470e-9ba2-b774d159bbe5`::Granted request >Thread-338::DEBUG::2014-05-13 12:35:45,479::task::827::TaskManager.Task::(resourceAcquired) Task=`ff3fca80-23af-4fe3-b6ec-b00ca27eda25`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:35:45,479::task::990::TaskManager.Task::(_decref) Task=`ff3fca80-23af-4fe3-b6ec-b00ca27eda25`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:35:45,479::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:35:45,479::task::1185::TaskManager.Task::(prepare) Task=`ff3fca80-23af-4fe3-b6ec-b00ca27eda25`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:35:45,480::task::595::TaskManager.Task::(_updateState) Task=`ff3fca80-23af-4fe3-b6ec-b00ca27eda25`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:35:45,480::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:35:45,480::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:35:45,480::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:35:45,480::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:35:45,480::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:35:45,481::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:35:45,481::task::990::TaskManager.Task::(_decref) Task=`ff3fca80-23af-4fe3-b6ec-b00ca27eda25`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:35:45,818::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:35:46,074::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:46,169::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0515219 s, 19.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:35:48,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:48,258::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321523 s, 31.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:35:48,837::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:35:50,271::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:50,351::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.03705 s, 27.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:35:51,795::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:35:51,803::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000762337 s, 5.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:35:51,856::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:35:51,881::task::595::TaskManager.Task::(_updateState) Task=`a6e1f731-564a-48eb-8506-2f76d34650aa`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:35:51,882::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:35:51,882::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000762337', 'lastCheck': '0.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:35:51,882::task::1185::TaskManager.Task::(prepare) Task=`a6e1f731-564a-48eb-8506-2f76d34650aa`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000762337', 'lastCheck': '0.1', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:35:51,882::task::595::TaskManager.Task::(_updateState) Task=`a6e1f731-564a-48eb-8506-2f76d34650aa`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:35:51,882::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:35:51,882::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:35:51,883::task::990::TaskManager.Task::(_decref) Task=`a6e1f731-564a-48eb-8506-2f76d34650aa`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:35:52,364::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:52,446::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384961 s, 26.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:35:54,460::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:54,544::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0398499 s, 25.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:35:54,876::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:35:55,499::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:35:55,500::task::595::TaskManager.Task::(_updateState) Task=`f5beae42-1c4a-4ed7-946b-34b6cee20738`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:35:55,500::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:35:55,500::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:35:55,501::task::1185::TaskManager.Task::(prepare) Task=`f5beae42-1c4a-4ed7-946b-34b6cee20738`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:35:55,501::task::595::TaskManager.Task::(_updateState) Task=`f5beae42-1c4a-4ed7-946b-34b6cee20738`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:35:55,501::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:35:55,501::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:35:55,501::task::990::TaskManager.Task::(_decref) Task=`f5beae42-1c4a-4ed7-946b-34b6cee20738`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:35:55,506::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:35:55,506::task::595::TaskManager.Task::(_updateState) Task=`4180f1df-7c4b-4f47-9ef8-3fd472ca66a8`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:35:55,506::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:35:55,507::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`81073e2b-d8a0-48d2-ad3b-91797f0e9a5c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:35:55,507::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:35:55,507::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:35:55,507::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`81073e2b-d8a0-48d2-ad3b-91797f0e9a5c`::Granted request >Thread-338::DEBUG::2014-05-13 12:35:55,508::task::827::TaskManager.Task::(resourceAcquired) Task=`4180f1df-7c4b-4f47-9ef8-3fd472ca66a8`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:35:55,509::task::990::TaskManager.Task::(_decref) Task=`4180f1df-7c4b-4f47-9ef8-3fd472ca66a8`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:35:55,509::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:35:55,510::task::1185::TaskManager.Task::(prepare) Task=`4180f1df-7c4b-4f47-9ef8-3fd472ca66a8`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:35:55,510::task::595::TaskManager.Task::(_updateState) Task=`4180f1df-7c4b-4f47-9ef8-3fd472ca66a8`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:35:55,510::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:35:55,510::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:35:55,510::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:35:55,511::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:35:55,511::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:35:55,511::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:35:55,511::task::990::TaskManager.Task::(_decref) Task=`4180f1df-7c4b-4f47-9ef8-3fd472ca66a8`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:35:56,557::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:56,648::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0466279 s, 22.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:35:57,895::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:35:58,662::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:35:58,736::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314579 s, 32.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:36:00,749::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:00,825::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315176 s, 32.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:36:00,913::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:36:01,811::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:36:01,819::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00135819 s, 3.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:36:02,839::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:02,919::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0375406 s, 27.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:36:03,931::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:36:04,933::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:05,015::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0378252 s, 27.1 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:36:05,533::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:36:05,534::task::595::TaskManager.Task::(_updateState) Task=`66e19643-175b-4fc8-b688-b27f108799a2`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:36:05,534::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:36:05,534::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:36:05,535::task::1185::TaskManager.Task::(prepare) Task=`66e19643-175b-4fc8-b688-b27f108799a2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:36:05,535::task::595::TaskManager.Task::(_updateState) Task=`66e19643-175b-4fc8-b688-b27f108799a2`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:36:05,535::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:36:05,535::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:36:05,535::task::990::TaskManager.Task::(_decref) Task=`66e19643-175b-4fc8-b688-b27f108799a2`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:36:05,540::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:36:05,541::task::595::TaskManager.Task::(_updateState) Task=`eb208d5b-9d5c-431f-9ce2-30fcb2038aa1`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:36:05,541::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:36:05,541::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`91f20d64-61b5-4735-934e-b636c6575064`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:36:05,541::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:36:05,542::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:36:05,542::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`91f20d64-61b5-4735-934e-b636c6575064`::Granted request >Thread-338::DEBUG::2014-05-13 12:36:05,542::task::827::TaskManager.Task::(resourceAcquired) Task=`eb208d5b-9d5c-431f-9ce2-30fcb2038aa1`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:36:05,542::task::990::TaskManager.Task::(_decref) Task=`eb208d5b-9d5c-431f-9ce2-30fcb2038aa1`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:36:05,543::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:36:05,543::task::1185::TaskManager.Task::(prepare) Task=`eb208d5b-9d5c-431f-9ce2-30fcb2038aa1`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:36:05,543::task::595::TaskManager.Task::(_updateState) Task=`eb208d5b-9d5c-431f-9ce2-30fcb2038aa1`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:36:05,544::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:36:05,544::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:36:05,544::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:36:05,544::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:36:05,544::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:36:05,545::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:36:05,545::task::990::TaskManager.Task::(_decref) Task=`eb208d5b-9d5c-431f-9ce2-30fcb2038aa1`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:36:06,954::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:36:07,018::task::595::TaskManager.Task::(_updateState) Task=`27178ab6-d767-44da-8678-d78cd5ebb57a`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:36:07,018::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:36:07,018::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00135819', 'lastCheck': '5.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:36:07,019::task::1185::TaskManager.Task::(prepare) Task=`27178ab6-d767-44da-8678-d78cd5ebb57a`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00135819', 'lastCheck': '5.2', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:36:07,019::task::595::TaskManager.Task::(_updateState) Task=`27178ab6-d767-44da-8678-d78cd5ebb57a`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:36:07,019::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:36:07,019::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:36:07,019::task::990::TaskManager.Task::(_decref) Task=`27178ab6-d767-44da-8678-d78cd5ebb57a`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:36:07,028::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:07,105::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321568 s, 31.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:36:09,119::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:09,539::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.375917 s, 2.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:36:09,974::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:36:11,552::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:11,652::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0376283 s, 27.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:36:11,827::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:36:11,834::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000715823 s, 5.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:36:12,993::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:36:13,665::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:13,740::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0324112 s, 31.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:36:15,563::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [30b3ccf] >Thread-13::DEBUG::2014-05-13 12:36:15,564::task::595::TaskManager.Task::(_updateState) Task=`d23825a8-e019-4878-8320-6134b2efa511`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:36:15,564::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:36:15,564::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:36:15,564::task::1185::TaskManager.Task::(prepare) Task=`d23825a8-e019-4878-8320-6134b2efa511`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:36:15,565::task::595::TaskManager.Task::(_updateState) Task=`d23825a8-e019-4878-8320-6134b2efa511`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:36:15,565::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:36:15,565::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:36:15,565::task::990::TaskManager.Task::(_decref) Task=`d23825a8-e019-4878-8320-6134b2efa511`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:36:15,569::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [30b3ccf] >Thread-338::DEBUG::2014-05-13 12:36:15,569::task::595::TaskManager.Task::(_updateState) Task=`c55ebaf2-c3e5-439f-97da-8f924cff9569`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:36:15,570::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:36:15,570::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8d25cc0a-35e7-4fea-8ca0-fce91d0c44ca`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:36:15,570::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:36:15,570::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:36:15,570::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8d25cc0a-35e7-4fea-8ca0-fce91d0c44ca`::Granted request >Thread-338::DEBUG::2014-05-13 12:36:15,571::task::827::TaskManager.Task::(resourceAcquired) Task=`c55ebaf2-c3e5-439f-97da-8f924cff9569`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:36:15,571::task::990::TaskManager.Task::(_decref) Task=`c55ebaf2-c3e5-439f-97da-8f924cff9569`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:36:15,572::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:36:15,572::task::1185::TaskManager.Task::(prepare) Task=`c55ebaf2-c3e5-439f-97da-8f924cff9569`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:36:15,572::task::595::TaskManager.Task::(_updateState) Task=`c55ebaf2-c3e5-439f-97da-8f924cff9569`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:36:15,572::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:36:15,572::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:36:15,572::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:36:15,573::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:36:15,573::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:36:15,573::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:36:15,573::task::990::TaskManager.Task::(_decref) Task=`c55ebaf2-c3e5-439f-97da-8f924cff9569`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:36:15,753::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:15,829::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322738 s, 31.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:36:16,012::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:36:17,843::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:17,924::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0380143 s, 26.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:36:19,032::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:36:19,937::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:20,017::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0369159 s, 27.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:36:21,841::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:36:21,849::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000594441 s, 6.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:36:22,029::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:36:22,054::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:36:22,114::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0412153 s, 24.8 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:36:22,158::task::595::TaskManager.Task::(_updateState) Task=`7b272f5c-63b6-485d-b5a2-2687c279ef67`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:36:22,159::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:36:22,159::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000594441', 'lastCheck': '0.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:36:22,159::task::1185::TaskManager.Task::(prepare) Task=`7b272f5c-63b6-485d-b5a2-2687c279ef67`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000594441', 'lastCheck': '0.3', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:36:22,159::task::595::TaskManager.Task::(_updateState) Task=`7b272f5c-63b6-485d-b5a2-2687c279ef67`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:36:22,160::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:36:22,160::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:36:22,160::task::990::TaskManager.Task::(_decref) Task=`7b272f5c-63b6-485d-b5a2-2687c279ef67`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:36:24,127::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:24,491::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.322202 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:36:25,073::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:36:25,590::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [72471f8f] >Thread-13::DEBUG::2014-05-13 12:36:25,590::task::595::TaskManager.Task::(_updateState) Task=`08f8a94e-d97a-498b-b041-f22ba2fd7ed5`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:36:25,591::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:36:25,591::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:36:25,591::task::1185::TaskManager.Task::(prepare) Task=`08f8a94e-d97a-498b-b041-f22ba2fd7ed5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:36:25,591::task::595::TaskManager.Task::(_updateState) Task=`08f8a94e-d97a-498b-b041-f22ba2fd7ed5`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:36:25,591::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:36:25,592::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:36:25,592::task::990::TaskManager.Task::(_decref) Task=`08f8a94e-d97a-498b-b041-f22ba2fd7ed5`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:36:25,597::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [72471f8f] >Thread-338::DEBUG::2014-05-13 12:36:25,597::task::595::TaskManager.Task::(_updateState) Task=`8fd20f4e-07a7-414b-83dc-543b8c589999`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:36:25,597::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:36:25,598::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b0c59484-5dc2-481f-85ba-c5b59a21ec83`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:36:25,598::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:36:25,598::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:36:25,598::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b0c59484-5dc2-481f-85ba-c5b59a21ec83`::Granted request >Thread-338::DEBUG::2014-05-13 12:36:25,599::task::827::TaskManager.Task::(resourceAcquired) Task=`8fd20f4e-07a7-414b-83dc-543b8c589999`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:36:25,599::task::990::TaskManager.Task::(_decref) Task=`8fd20f4e-07a7-414b-83dc-543b8c589999`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:36:25,600::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:36:25,600::task::1185::TaskManager.Task::(prepare) Task=`8fd20f4e-07a7-414b-83dc-543b8c589999`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:36:25,600::task::595::TaskManager.Task::(_updateState) Task=`8fd20f4e-07a7-414b-83dc-543b8c589999`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:36:25,600::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:36:25,600::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:36:25,601::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:36:25,601::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:36:25,601::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:36:25,601::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:36:25,601::task::990::TaskManager.Task::(_decref) Task=`8fd20f4e-07a7-414b-83dc-543b8c589999`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:36:26,504::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:26,585::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0371601 s, 27.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:36:28,092::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:36:28,598::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:28,672::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314984 s, 32.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:36:30,685::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:30,760::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315934 s, 32.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:36:31,110::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:36:31,856::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:36:31,864::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000644634 s, 6.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:36:32,774::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:32,854::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374133 s, 27.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:36:34,128::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:36:34,867::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:34,947::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037103 s, 27.6 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:36:35,622::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [7d725256] >Thread-13::DEBUG::2014-05-13 12:36:35,622::task::595::TaskManager.Task::(_updateState) Task=`e8d78539-b792-4522-8387-efffb495d43b`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:36:35,622::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:36:35,623::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:36:35,623::task::1185::TaskManager.Task::(prepare) Task=`e8d78539-b792-4522-8387-efffb495d43b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:36:35,623::task::595::TaskManager.Task::(_updateState) Task=`e8d78539-b792-4522-8387-efffb495d43b`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:36:35,623::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:36:35,623::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:36:35,624::task::990::TaskManager.Task::(_decref) Task=`e8d78539-b792-4522-8387-efffb495d43b`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:36:35,629::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [7d725256] >Thread-338::DEBUG::2014-05-13 12:36:35,629::task::595::TaskManager.Task::(_updateState) Task=`60daa10d-1dd3-44a5-9e84-aba2e3f74b74`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:36:35,630::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:36:35,630::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`658faaef-1205-4e1c-9dd9-8df037e5a1fa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:36:35,630::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:36:35,630::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:36:35,631::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`658faaef-1205-4e1c-9dd9-8df037e5a1fa`::Granted request >Thread-338::DEBUG::2014-05-13 12:36:35,631::task::827::TaskManager.Task::(resourceAcquired) Task=`60daa10d-1dd3-44a5-9e84-aba2e3f74b74`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:36:35,631::task::990::TaskManager.Task::(_decref) Task=`60daa10d-1dd3-44a5-9e84-aba2e3f74b74`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:36:35,632::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:36:35,632::task::1185::TaskManager.Task::(prepare) Task=`60daa10d-1dd3-44a5-9e84-aba2e3f74b74`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:36:35,632::task::595::TaskManager.Task::(_updateState) Task=`60daa10d-1dd3-44a5-9e84-aba2e3f74b74`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:36:35,632::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:36:35,633::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:36:35,633::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:36:35,633::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:36:35,633::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:36:35,633::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:36:35,634::task::990::TaskManager.Task::(_decref) Task=`60daa10d-1dd3-44a5-9e84-aba2e3f74b74`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:36:36,960::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:37,046::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0438467 s, 23.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:36:37,147::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:36:37,293::task::595::TaskManager.Task::(_updateState) Task=`e67ea259-f06d-4070-a73a-b47976a19595`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:36:37,293::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:36:37,293::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000644634', 'lastCheck': '5.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:36:37,294::task::1185::TaskManager.Task::(prepare) Task=`e67ea259-f06d-4070-a73a-b47976a19595`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000644634', 'lastCheck': '5.4', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:36:37,294::task::595::TaskManager.Task::(_updateState) Task=`e67ea259-f06d-4070-a73a-b47976a19595`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:36:37,294::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:36:37,294::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:36:37,294::task::990::TaskManager.Task::(_decref) Task=`e67ea259-f06d-4070-a73a-b47976a19595`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:36:39,060::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:39,136::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0341085 s, 30.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:36:40,166::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:36:41,149::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:41,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383798 s, 26.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:36:41,871::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:36:41,879::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00065365 s, 6.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:36:43,187::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:36:43,242::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:43,324::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0390434 s, 26.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:36:45,336::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:45,414::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0342012 s, 29.9 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:36:45,652::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:36:45,652::task::595::TaskManager.Task::(_updateState) Task=`8dc46351-b308-493e-96ad-3ff0231c872c`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:36:45,653::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:36:45,653::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:36:45,653::task::1185::TaskManager.Task::(prepare) Task=`8dc46351-b308-493e-96ad-3ff0231c872c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:36:45,653::task::595::TaskManager.Task::(_updateState) Task=`8dc46351-b308-493e-96ad-3ff0231c872c`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:36:45,654::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:36:45,654::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:36:45,654::task::990::TaskManager.Task::(_decref) Task=`8dc46351-b308-493e-96ad-3ff0231c872c`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:36:45,658::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:36:45,659::task::595::TaskManager.Task::(_updateState) Task=`79136524-d3cc-4f72-a595-c9c4d9f120b7`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:36:45,659::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:36:45,659::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f3af7b38-ab80-43f8-a53f-27b964af718c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:36:45,660::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:36:45,660::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:36:45,660::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f3af7b38-ab80-43f8-a53f-27b964af718c`::Granted request >Thread-338::DEBUG::2014-05-13 12:36:45,660::task::827::TaskManager.Task::(resourceAcquired) Task=`79136524-d3cc-4f72-a595-c9c4d9f120b7`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:36:45,660::task::990::TaskManager.Task::(_decref) Task=`79136524-d3cc-4f72-a595-c9c4d9f120b7`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:36:45,661::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:36:45,661::task::1185::TaskManager.Task::(prepare) Task=`79136524-d3cc-4f72-a595-c9c4d9f120b7`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:36:45,662::task::595::TaskManager.Task::(_updateState) Task=`79136524-d3cc-4f72-a595-c9c4d9f120b7`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:36:45,662::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:36:45,662::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:36:45,662::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:36:45,662::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:36:45,663::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:36:45,663::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:36:45,663::task::990::TaskManager.Task::(_decref) Task=`79136524-d3cc-4f72-a595-c9c4d9f120b7`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:36:46,206::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:36:47,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:47,507::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0369177 s, 27.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:36:49,223::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:36:49,520::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:49,610::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0473296 s, 21.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:36:51,624::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:51,698::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306345 s, 33.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:36:51,887::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:36:51,894::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000669109 s, 6.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:36:52,241::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:36:52,432::task::595::TaskManager.Task::(_updateState) Task=`642e0f53-30ad-4579-8892-4b08fd049f3f`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:36:52,432::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:36:52,433::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000669109', 'lastCheck': '0.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:36:52,433::task::1185::TaskManager.Task::(prepare) Task=`642e0f53-30ad-4579-8892-4b08fd049f3f`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000669109', 'lastCheck': '0.5', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:36:52,433::task::595::TaskManager.Task::(_updateState) Task=`642e0f53-30ad-4579-8892-4b08fd049f3f`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:36:52,433::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:36:52,433::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:36:52,433::task::990::TaskManager.Task::(_decref) Task=`642e0f53-30ad-4579-8892-4b08fd049f3f`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:36:53,711::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:53,797::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0412603 s, 24.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:36:55,261::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:36:55,681::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:36:55,681::task::595::TaskManager.Task::(_updateState) Task=`37416369-dc98-4572-9bde-2002536bf886`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:36:55,682::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:36:55,682::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:36:55,682::task::1185::TaskManager.Task::(prepare) Task=`37416369-dc98-4572-9bde-2002536bf886`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:36:55,682::task::595::TaskManager.Task::(_updateState) Task=`37416369-dc98-4572-9bde-2002536bf886`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:36:55,683::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:36:55,683::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:36:55,683::task::990::TaskManager.Task::(_decref) Task=`37416369-dc98-4572-9bde-2002536bf886`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:36:55,687::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:36:55,687::task::595::TaskManager.Task::(_updateState) Task=`2d42006e-774e-4032-bd5e-2a1da3cc1057`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:36:55,688::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:36:55,688::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`34a19983-5eb7-46c7-b2c0-dbf1e76a46f3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:36:55,688::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:36:55,689::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:36:55,689::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`34a19983-5eb7-46c7-b2c0-dbf1e76a46f3`::Granted request >Thread-338::DEBUG::2014-05-13 12:36:55,689::task::827::TaskManager.Task::(resourceAcquired) Task=`2d42006e-774e-4032-bd5e-2a1da3cc1057`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:36:55,689::task::990::TaskManager.Task::(_decref) Task=`2d42006e-774e-4032-bd5e-2a1da3cc1057`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:36:55,690::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:36:55,690::task::1185::TaskManager.Task::(prepare) Task=`2d42006e-774e-4032-bd5e-2a1da3cc1057`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:36:55,690::task::595::TaskManager.Task::(_updateState) Task=`2d42006e-774e-4032-bd5e-2a1da3cc1057`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:36:55,691::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:36:55,691::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:36:55,691::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:36:55,691::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:36:55,691::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:36:55,691::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:36:55,692::task::990::TaskManager.Task::(_decref) Task=`2d42006e-774e-4032-bd5e-2a1da3cc1057`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:36:55,810::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:55,889::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364032 s, 28.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:36:57,902::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:36:57,978::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0339103 s, 30.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:36:58,286::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:36:59,991::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:00,069::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379323 s, 27.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:37:01,315::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:37:01,901::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:37:01,909::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000668665 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:37:02,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:02,158::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0389037 s, 26.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:37:04,171::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:04,246::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0340528 s, 30.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:37:04,337::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:37:05,715::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:37:05,716::task::595::TaskManager.Task::(_updateState) Task=`e9d44bfb-7af7-4cd1-85cc-3553a1a3905d`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:37:05,716::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:37:05,716::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:37:05,716::task::1185::TaskManager.Task::(prepare) Task=`e9d44bfb-7af7-4cd1-85cc-3553a1a3905d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:37:05,717::task::595::TaskManager.Task::(_updateState) Task=`e9d44bfb-7af7-4cd1-85cc-3553a1a3905d`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:37:05,717::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:37:05,717::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:37:05,717::task::990::TaskManager.Task::(_decref) Task=`e9d44bfb-7af7-4cd1-85cc-3553a1a3905d`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:37:05,722::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:37:05,723::task::595::TaskManager.Task::(_updateState) Task=`39b20229-e0d7-41aa-87e8-dee723817c62`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:37:05,723::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:37:05,723::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`03768495-10e4-4b79-aa7e-f0d93b6fbced`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:37:05,724::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:37:05,724::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:37:05,724::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`03768495-10e4-4b79-aa7e-f0d93b6fbced`::Granted request >Thread-338::DEBUG::2014-05-13 12:37:05,725::task::827::TaskManager.Task::(resourceAcquired) Task=`39b20229-e0d7-41aa-87e8-dee723817c62`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:37:05,725::task::990::TaskManager.Task::(_decref) Task=`39b20229-e0d7-41aa-87e8-dee723817c62`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:37:05,726::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:37:05,726::task::1185::TaskManager.Task::(prepare) Task=`39b20229-e0d7-41aa-87e8-dee723817c62`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:37:05,727::task::595::TaskManager.Task::(_updateState) Task=`39b20229-e0d7-41aa-87e8-dee723817c62`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:37:05,727::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:37:05,727::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:37:05,727::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:37:05,727::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:37:05,728::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:37:05,728::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:37:05,728::task::990::TaskManager.Task::(_decref) Task=`39b20229-e0d7-41aa-87e8-dee723817c62`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:37:06,259::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:06,471::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0447802 s, 22.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:37:07,358::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:37:07,574::task::595::TaskManager.Task::(_updateState) Task=`8797449d-1359-4e33-a092-f639a0871313`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:37:07,574::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:37:07,574::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000668665', 'lastCheck': '5.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:37:07,574::task::1185::TaskManager.Task::(prepare) Task=`8797449d-1359-4e33-a092-f639a0871313`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000668665', 'lastCheck': '5.7', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:37:07,574::task::595::TaskManager.Task::(_updateState) Task=`8797449d-1359-4e33-a092-f639a0871313`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:37:07,575::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:37:07,575::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:37:07,575::task::990::TaskManager.Task::(_decref) Task=`8797449d-1359-4e33-a092-f639a0871313`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:37:08,511::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:08,593::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0388381 s, 26.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:37:10,376::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:37:10,606::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:10,694::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0389589 s, 26.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:37:11,916::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:37:11,924::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000724369 s, 5.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:37:12,706::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:12,783::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0338936 s, 30.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:37:13,401::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:37:14,796::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:14,877::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0385971 s, 26.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:37:15,746::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4765f2fb] >Thread-13::DEBUG::2014-05-13 12:37:15,746::task::595::TaskManager.Task::(_updateState) Task=`9bdd8f3e-e66d-415b-af03-986740c3631b`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:37:15,747::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:37:15,747::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:37:15,747::task::1185::TaskManager.Task::(prepare) Task=`9bdd8f3e-e66d-415b-af03-986740c3631b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:37:15,747::task::595::TaskManager.Task::(_updateState) Task=`9bdd8f3e-e66d-415b-af03-986740c3631b`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:37:15,747::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:37:15,748::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:37:15,748::task::990::TaskManager.Task::(_decref) Task=`9bdd8f3e-e66d-415b-af03-986740c3631b`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:37:15,752::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4765f2fb] >Thread-338::DEBUG::2014-05-13 12:37:15,752::task::595::TaskManager.Task::(_updateState) Task=`24544164-c588-46b0-bd40-de315cb4715c`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:37:15,752::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:37:15,753::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a1e46b31-05cd-4c0b-bd88-0b6ab01f2553`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:37:15,753::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:37:15,753::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:37:15,753::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a1e46b31-05cd-4c0b-bd88-0b6ab01f2553`::Granted request >Thread-338::DEBUG::2014-05-13 12:37:15,754::task::827::TaskManager.Task::(resourceAcquired) Task=`24544164-c588-46b0-bd40-de315cb4715c`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:37:15,754::task::990::TaskManager.Task::(_decref) Task=`24544164-c588-46b0-bd40-de315cb4715c`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:37:15,755::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:37:15,755::task::1185::TaskManager.Task::(prepare) Task=`24544164-c588-46b0-bd40-de315cb4715c`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:37:15,755::task::595::TaskManager.Task::(_updateState) Task=`24544164-c588-46b0-bd40-de315cb4715c`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:37:15,755::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:37:15,755::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:37:15,755::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:37:15,756::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:37:15,756::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:37:15,756::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:37:15,756::task::990::TaskManager.Task::(_decref) Task=`24544164-c588-46b0-bd40-de315cb4715c`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:37:16,421::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:37:16,890::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:16,965::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315526 s, 32.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:37:18,978::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:19,054::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0337269 s, 30.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:37:19,440::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:37:21,067::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:21,156::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0449064 s, 22.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:37:21,931::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:37:21,939::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000729728 s, 5.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:37:22,459::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:37:22,707::task::595::TaskManager.Task::(_updateState) Task=`c2cc817f-9f80-4125-85a2-488f7365be54`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:37:22,707::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:37:22,708::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000729728', 'lastCheck': '0.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:37:22,708::task::1185::TaskManager.Task::(prepare) Task=`c2cc817f-9f80-4125-85a2-488f7365be54`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000729728', 'lastCheck': '0.8', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:37:22,708::task::595::TaskManager.Task::(_updateState) Task=`c2cc817f-9f80-4125-85a2-488f7365be54`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:37:22,708::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:37:22,708::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:37:22,708::task::990::TaskManager.Task::(_decref) Task=`c2cc817f-9f80-4125-85a2-488f7365be54`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:37:23,169::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:23,250::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364951 s, 28.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:37:25,263::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:25,343::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037149 s, 27.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:37:25,484::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:37:25,774::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:37:25,775::task::595::TaskManager.Task::(_updateState) Task=`9a468831-a2ee-4278-9388-11593ed0ebfe`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:37:25,775::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:37:25,775::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:37:25,775::task::1185::TaskManager.Task::(prepare) Task=`9a468831-a2ee-4278-9388-11593ed0ebfe`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:37:25,776::task::595::TaskManager.Task::(_updateState) Task=`9a468831-a2ee-4278-9388-11593ed0ebfe`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:37:25,776::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:37:25,776::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:37:25,776::task::990::TaskManager.Task::(_decref) Task=`9a468831-a2ee-4278-9388-11593ed0ebfe`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:37:25,781::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:37:25,781::task::595::TaskManager.Task::(_updateState) Task=`21961bec-a5bc-4321-b3b6-a8d7f1b0b32e`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:37:25,781::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:37:25,782::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`58c8f81e-4b52-4994-b7bd-7ffb3cf76457`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:37:25,782::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:37:25,782::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:37:25,782::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`58c8f81e-4b52-4994-b7bd-7ffb3cf76457`::Granted request >Thread-338::DEBUG::2014-05-13 12:37:25,783::task::827::TaskManager.Task::(resourceAcquired) Task=`21961bec-a5bc-4321-b3b6-a8d7f1b0b32e`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:37:25,783::task::990::TaskManager.Task::(_decref) Task=`21961bec-a5bc-4321-b3b6-a8d7f1b0b32e`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:37:25,783::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:37:25,784::task::1185::TaskManager.Task::(prepare) Task=`21961bec-a5bc-4321-b3b6-a8d7f1b0b32e`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:37:25,784::task::595::TaskManager.Task::(_updateState) Task=`21961bec-a5bc-4321-b3b6-a8d7f1b0b32e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:37:25,784::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:37:25,784::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:37:25,784::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:37:25,785::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:37:25,785::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:37:25,785::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:37:25,785::task::990::TaskManager.Task::(_decref) Task=`21961bec-a5bc-4321-b3b6-a8d7f1b0b32e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:37:27,356::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:27,432::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323738 s, 31.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:37:28,513::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:37:29,445::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:29,525::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374808 s, 27.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:37:31,532::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:37:31,538::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:31,612::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311044 s, 32.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:37:31,946::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:37:31,954::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000705527 s, 5.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:37:33,625::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:34,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.364425 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:37:34,553::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:37:35,814::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:37:35,814::task::595::TaskManager.Task::(_updateState) Task=`9e2f64a2-9abb-4bc9-b355-3a7c2d46ee12`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:37:35,815::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:37:35,815::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:37:35,815::task::1185::TaskManager.Task::(prepare) Task=`9e2f64a2-9abb-4bc9-b355-3a7c2d46ee12`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:37:35,815::task::595::TaskManager.Task::(_updateState) Task=`9e2f64a2-9abb-4bc9-b355-3a7c2d46ee12`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:37:35,815::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:37:35,816::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:37:35,816::task::990::TaskManager.Task::(_decref) Task=`9e2f64a2-9abb-4bc9-b355-3a7c2d46ee12`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:37:35,823::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:37:35,824::task::595::TaskManager.Task::(_updateState) Task=`52985347-573d-4011-8657-c0da0975a9f2`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:37:35,824::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:37:35,824::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`54ba415f-f5d8-43f8-833b-ad42e0af479e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:37:35,825::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:37:35,825::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:37:35,825::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`54ba415f-f5d8-43f8-833b-ad42e0af479e`::Granted request >Thread-338::DEBUG::2014-05-13 12:37:35,825::task::827::TaskManager.Task::(resourceAcquired) Task=`52985347-573d-4011-8657-c0da0975a9f2`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:37:35,826::task::990::TaskManager.Task::(_decref) Task=`52985347-573d-4011-8657-c0da0975a9f2`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:37:35,826::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:37:35,826::task::1185::TaskManager.Task::(prepare) Task=`52985347-573d-4011-8657-c0da0975a9f2`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:37:35,827::task::595::TaskManager.Task::(_updateState) Task=`52985347-573d-4011-8657-c0da0975a9f2`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:37:35,827::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:37:35,827::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:37:35,827::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:37:35,827::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:37:35,828::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:37:35,828::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:37:35,828::task::990::TaskManager.Task::(_decref) Task=`52985347-573d-4011-8657-c0da0975a9f2`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:37:36,046::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:36,120::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382339 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:37:37,573::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:37:37,894::task::595::TaskManager.Task::(_updateState) Task=`375d090b-5255-4cd6-be1d-249f75866052`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:37:37,894::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:37:37,894::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000705527', 'lastCheck': '5.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:37:37,894::task::1185::TaskManager.Task::(prepare) Task=`375d090b-5255-4cd6-be1d-249f75866052`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000705527', 'lastCheck': '5.9', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:37:37,894::task::595::TaskManager.Task::(_updateState) Task=`375d090b-5255-4cd6-be1d-249f75866052`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:37:37,895::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:37:37,895::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:37:37,895::task::990::TaskManager.Task::(_decref) Task=`375d090b-5255-4cd6-be1d-249f75866052`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:37:38,128::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:38,203::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0337345 s, 30.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:37:40,216::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:40,296::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0386015 s, 26.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:37:40,592::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:37:41,961::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:37:41,969::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000641461 s, 6.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:37:42,309::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:42,384::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0337686 s, 30.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:37:43,611::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:37:44,397::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:44,478::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.038644 s, 26.5 MB/s\n'; <rc> = 0 >Thread-13::DEBUG::2014-05-13 12:37:45,851::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:37:45,852::task::595::TaskManager.Task::(_updateState) Task=`f1cb2c69-c05f-46be-8127-02989b7a95f8`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:37:45,852::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:37:45,852::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:37:45,852::task::1185::TaskManager.Task::(prepare) Task=`f1cb2c69-c05f-46be-8127-02989b7a95f8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:37:45,852::task::595::TaskManager.Task::(_updateState) Task=`f1cb2c69-c05f-46be-8127-02989b7a95f8`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:37:45,853::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:37:45,853::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:37:45,853::task::990::TaskManager.Task::(_decref) Task=`f1cb2c69-c05f-46be-8127-02989b7a95f8`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:37:45,857::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:37:45,858::task::595::TaskManager.Task::(_updateState) Task=`b7a3ea50-c7b1-4b07-9bd4-894696a10c84`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:37:45,858::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:37:45,859::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e5aedb0b-3ebd-4664-9f4f-630865354753`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:37:45,859::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:37:45,860::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:37:45,860::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e5aedb0b-3ebd-4664-9f4f-630865354753`::Granted request >Thread-338::DEBUG::2014-05-13 12:37:45,860::task::827::TaskManager.Task::(resourceAcquired) Task=`b7a3ea50-c7b1-4b07-9bd4-894696a10c84`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:37:45,861::task::990::TaskManager.Task::(_decref) Task=`b7a3ea50-c7b1-4b07-9bd4-894696a10c84`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:37:45,862::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:37:45,862::task::1185::TaskManager.Task::(prepare) Task=`b7a3ea50-c7b1-4b07-9bd4-894696a10c84`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:37:45,862::task::595::TaskManager.Task::(_updateState) Task=`b7a3ea50-c7b1-4b07-9bd4-894696a10c84`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:37:45,863::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:37:45,863::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:37:45,863::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:37:45,864::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:37:45,864::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:37:45,864::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:37:45,865::task::990::TaskManager.Task::(_decref) Task=`b7a3ea50-c7b1-4b07-9bd4-894696a10c84`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:37:46,491::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:46,566::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0334522 s, 30.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:37:46,630::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:37:48,579::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:48,946::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.324778 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:37:49,669::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:37:50,959::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:51,040::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0390203 s, 26.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:37:51,976::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:37:51,983::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000537601 s, 7.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:37:52,713::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:37:53,033::task::595::TaskManager.Task::(_updateState) Task=`4729254b-ce45-4179-be47-3ed71a371990`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:37:53,033::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-13::INFO::2014-05-13 12:37:53,034::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000537601', 'lastCheck': '1.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:37:53,034::task::1185::TaskManager.Task::(prepare) Task=`4729254b-ce45-4179-be47-3ed71a371990`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000537601', 'lastCheck': '1.0', 'valid': True}} >Thread-13::DEBUG::2014-05-13 12:37:53,034::task::595::TaskManager.Task::(_updateState) Task=`4729254b-ce45-4179-be47-3ed71a371990`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:37:53,034::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:37:53,034::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:37:53,035::task::990::TaskManager.Task::(_decref) Task=`4729254b-ce45-4179-be47-3ed71a371990`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:37:53,053::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:53,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382261 s, 26.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:37:55,148::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:55,231::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0393743 s, 26.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:37:55,739::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-13::DEBUG::2014-05-13 12:37:55,885::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:37:55,885::task::595::TaskManager.Task::(_updateState) Task=`1bf5957a-ee63-441c-a91c-3bee06d36b33`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:37:55,885::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:37:55,886::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:37:55,886::task::1185::TaskManager.Task::(prepare) Task=`1bf5957a-ee63-441c-a91c-3bee06d36b33`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:37:55,886::task::595::TaskManager.Task::(_updateState) Task=`1bf5957a-ee63-441c-a91c-3bee06d36b33`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:37:55,886::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:37:55,886::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:37:55,887::task::990::TaskManager.Task::(_decref) Task=`1bf5957a-ee63-441c-a91c-3bee06d36b33`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:37:56,318::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:37:56,319::task::595::TaskManager.Task::(_updateState) Task=`31e1d74b-bad8-4b32-830f-72db30b01407`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:37:56,319::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:37:56,319::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c8a3ce97-b6dc-4dd4-89e2-d2bd3973bb10`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:37:56,320::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:37:56,320::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:37:56,320::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c8a3ce97-b6dc-4dd4-89e2-d2bd3973bb10`::Granted request >Thread-338::DEBUG::2014-05-13 12:37:56,320::task::827::TaskManager.Task::(resourceAcquired) Task=`31e1d74b-bad8-4b32-830f-72db30b01407`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:37:56,321::task::990::TaskManager.Task::(_decref) Task=`31e1d74b-bad8-4b32-830f-72db30b01407`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:37:56,321::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:37:56,322::task::1185::TaskManager.Task::(prepare) Task=`31e1d74b-bad8-4b32-830f-72db30b01407`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:37:56,322::task::595::TaskManager.Task::(_updateState) Task=`31e1d74b-bad8-4b32-830f-72db30b01407`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:37:56,322::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:37:56,322::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:37:56,322::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:37:56,323::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:37:56,323::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:37:56,323::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:37:56,323::task::990::TaskManager.Task::(_decref) Task=`31e1d74b-bad8-4b32-830f-72db30b01407`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:37:57,244::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:57,332::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0462614 s, 22.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:37:58,757::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:37:59,345::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:37:59,426::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.038793 s, 26.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:38:01,439::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:01,514::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0333949 s, 30.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:38:01,774::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:38:01,991::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:38:01,999::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000619331 s, 6.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:38:03,527::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:03,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0325726 s, 31.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:38:04,795::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:38:05,615::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:05,695::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382272 s, 26.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:38:06,340::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:38:06,341::task::595::TaskManager.Task::(_updateState) Task=`983d2444-2395-425c-84a9-6757852db274`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:38:06,341::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:38:06,341::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:38:06,341::task::1185::TaskManager.Task::(prepare) Task=`983d2444-2395-425c-84a9-6757852db274`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:38:06,342::task::595::TaskManager.Task::(_updateState) Task=`983d2444-2395-425c-84a9-6757852db274`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:38:06,342::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:38:06,342::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:38:06,342::task::990::TaskManager.Task::(_decref) Task=`983d2444-2395-425c-84a9-6757852db274`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:38:06,346::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:38:06,347::task::595::TaskManager.Task::(_updateState) Task=`d26875e8-5989-4310-ae62-f7e10ac09ad6`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:38:06,347::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:38:06,347::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d09b3263-e61c-4387-901e-81c904d14e2a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:38:06,348::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:38:06,348::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:38:06,348::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d09b3263-e61c-4387-901e-81c904d14e2a`::Granted request >Thread-338::DEBUG::2014-05-13 12:38:06,348::task::827::TaskManager.Task::(resourceAcquired) Task=`d26875e8-5989-4310-ae62-f7e10ac09ad6`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:38:06,348::task::990::TaskManager.Task::(_decref) Task=`d26875e8-5989-4310-ae62-f7e10ac09ad6`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:38:06,349::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:38:06,349::task::1185::TaskManager.Task::(prepare) Task=`d26875e8-5989-4310-ae62-f7e10ac09ad6`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:38:06,349::task::595::TaskManager.Task::(_updateState) Task=`d26875e8-5989-4310-ae62-f7e10ac09ad6`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:38:06,350::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:38:06,350::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:38:06,350::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:38:06,350::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:38:06,350::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:38:06,350::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:38:06,351::task::990::TaskManager.Task::(_decref) Task=`d26875e8-5989-4310-ae62-f7e10ac09ad6`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:38:07,708::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:07,782::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309205 s, 33.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:38:07,814::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:38:08,383::task::595::TaskManager.Task::(_updateState) Task=`646cb1b3-bcef-4694-9d6c-ab6f97e55de4`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:38:08,383::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:38:08,384::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000619331', 'lastCheck': '6.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:38:08,384::task::1185::TaskManager.Task::(prepare) Task=`646cb1b3-bcef-4694-9d6c-ab6f97e55de4`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000619331', 'lastCheck': '6.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:38:08,384::task::595::TaskManager.Task::(_updateState) Task=`646cb1b3-bcef-4694-9d6c-ab6f97e55de4`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:38:08,384::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:38:08,384::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:38:08,385::task::990::TaskManager.Task::(_decref) Task=`646cb1b3-bcef-4694-9d6c-ab6f97e55de4`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:38:09,795::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:10,206::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.369587 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:38:10,835::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:38:12,006::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:38:12,014::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000704446 s, 5.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:38:12,219::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:12,294::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320514 s, 31.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:38:13,855::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:38:14,307::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:14,388::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0377799 s, 27.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:38:16,373::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:38:16,373::task::595::TaskManager.Task::(_updateState) Task=`dea83ac5-6a23-4aec-bb78-fd5d97dc50e4`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:38:16,374::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:38:16,374::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:38:16,374::task::1185::TaskManager.Task::(prepare) Task=`dea83ac5-6a23-4aec-bb78-fd5d97dc50e4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:38:16,374::task::595::TaskManager.Task::(_updateState) Task=`dea83ac5-6a23-4aec-bb78-fd5d97dc50e4`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:38:16,374::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:38:16,375::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:38:16,375::task::990::TaskManager.Task::(_decref) Task=`dea83ac5-6a23-4aec-bb78-fd5d97dc50e4`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:38:16,380::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:38:16,380::task::595::TaskManager.Task::(_updateState) Task=`36104b4f-1fb4-4a03-bcf8-7ed7b224c46c`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:38:16,380::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:38:16,380::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6f63471c-ac95-438d-9423-25a9c95f113d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:38:16,381::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:38:16,382::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:38:16,382::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6f63471c-ac95-438d-9423-25a9c95f113d`::Granted request >Thread-338::DEBUG::2014-05-13 12:38:16,382::task::827::TaskManager.Task::(resourceAcquired) Task=`36104b4f-1fb4-4a03-bcf8-7ed7b224c46c`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:38:16,382::task::990::TaskManager.Task::(_decref) Task=`36104b4f-1fb4-4a03-bcf8-7ed7b224c46c`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:38:16,383::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:38:16,383::task::1185::TaskManager.Task::(prepare) Task=`36104b4f-1fb4-4a03-bcf8-7ed7b224c46c`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:38:16,383::task::595::TaskManager.Task::(_updateState) Task=`36104b4f-1fb4-4a03-bcf8-7ed7b224c46c`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:38:16,384::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:38:16,384::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:38:16,384::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:38:16,384::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:38:16,384::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:38:16,384::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:38:16,385::task::990::TaskManager.Task::(_decref) Task=`36104b4f-1fb4-4a03-bcf8-7ed7b224c46c`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:38:16,401::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:16,478::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320429 s, 32.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:38:16,874::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:38:18,491::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:18,567::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320974 s, 31.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:38:19,894::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:38:20,580::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:20,661::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381788 s, 26.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:38:22,020::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:38:22,028::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000644712 s, 6.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:38:22,674::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:22,754::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384823 s, 26.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:38:22,913::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:38:23,517::task::595::TaskManager.Task::(_updateState) Task=`52b2178b-7639-4352-9218-6ae87e6e1f8c`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:38:23,517::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:38:23,518::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000644712', 'lastCheck': '1.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:38:23,518::task::1185::TaskManager.Task::(prepare) Task=`52b2178b-7639-4352-9218-6ae87e6e1f8c`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000644712', 'lastCheck': '1.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:38:23,518::task::595::TaskManager.Task::(_updateState) Task=`52b2178b-7639-4352-9218-6ae87e6e1f8c`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:38:23,518::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:38:23,518::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:38:23,518::task::990::TaskManager.Task::(_decref) Task=`52b2178b-7639-4352-9218-6ae87e6e1f8c`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:38:24,767::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:24,855::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0436652 s, 23.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:38:25,930::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:38:26,405::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [303c64a0] >Thread-66::DEBUG::2014-05-13 12:38:26,405::task::595::TaskManager.Task::(_updateState) Task=`7613d6b4-4b2e-4d55-882b-e3a9b33bd36d`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:38:26,405::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:38:26,406::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:38:26,406::task::1185::TaskManager.Task::(prepare) Task=`7613d6b4-4b2e-4d55-882b-e3a9b33bd36d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:38:26,406::task::595::TaskManager.Task::(_updateState) Task=`7613d6b4-4b2e-4d55-882b-e3a9b33bd36d`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:38:26,406::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:38:26,406::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:38:26,407::task::990::TaskManager.Task::(_decref) Task=`7613d6b4-4b2e-4d55-882b-e3a9b33bd36d`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:38:26,411::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [303c64a0] >Thread-338::DEBUG::2014-05-13 12:38:26,411::task::595::TaskManager.Task::(_updateState) Task=`f0373f36-f61a-41fd-81d3-67655a847184`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:38:26,411::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:38:26,412::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c659a5ec-b909-4b5a-9d9c-ea3a078cd17f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:38:26,412::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:38:26,412::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:38:26,413::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c659a5ec-b909-4b5a-9d9c-ea3a078cd17f`::Granted request >Thread-338::DEBUG::2014-05-13 12:38:26,413::task::827::TaskManager.Task::(resourceAcquired) Task=`f0373f36-f61a-41fd-81d3-67655a847184`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:38:26,413::task::990::TaskManager.Task::(_decref) Task=`f0373f36-f61a-41fd-81d3-67655a847184`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:38:26,414::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:38:26,414::task::1185::TaskManager.Task::(prepare) Task=`f0373f36-f61a-41fd-81d3-67655a847184`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:38:26,414::task::595::TaskManager.Task::(_updateState) Task=`f0373f36-f61a-41fd-81d3-67655a847184`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:38:26,414::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:38:26,415::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:38:26,415::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:38:26,415::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:38:26,415::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:38:26,415::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:38:26,416::task::990::TaskManager.Task::(_decref) Task=`f0373f36-f61a-41fd-81d3-67655a847184`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:38:26,868::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:26,949::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0386637 s, 26.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:38:28,951::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:38:28,962::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:29,042::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037732 s, 27.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:38:31,055::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:31,132::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0340401 s, 30.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:38:31,970::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:38:32,035::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:38:32,043::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000728436 s, 5.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:38:33,145::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:33,219::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0330005 s, 31.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:38:34,987::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:38:35,232::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:35,312::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0385506 s, 26.6 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:38:36,434::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:38:36,434::task::595::TaskManager.Task::(_updateState) Task=`26625cdf-9d98-4e8c-8e75-8a86b8d94259`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:38:36,434::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:38:36,435::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:38:36,435::task::1185::TaskManager.Task::(prepare) Task=`26625cdf-9d98-4e8c-8e75-8a86b8d94259`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:38:36,435::task::595::TaskManager.Task::(_updateState) Task=`26625cdf-9d98-4e8c-8e75-8a86b8d94259`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:38:36,435::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:38:36,435::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:38:36,436::task::990::TaskManager.Task::(_decref) Task=`26625cdf-9d98-4e8c-8e75-8a86b8d94259`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:38:36,440::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:38:36,440::task::595::TaskManager.Task::(_updateState) Task=`36670388-67cf-48b0-adf7-2cc215a3bb1c`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:38:36,440::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:38:36,441::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ecad8a16-4c60-498d-a5b4-1d31533ac021`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:38:36,441::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:38:36,441::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:38:36,441::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ecad8a16-4c60-498d-a5b4-1d31533ac021`::Granted request >Thread-338::DEBUG::2014-05-13 12:38:36,442::task::827::TaskManager.Task::(resourceAcquired) Task=`36670388-67cf-48b0-adf7-2cc215a3bb1c`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:38:36,442::task::990::TaskManager.Task::(_decref) Task=`36670388-67cf-48b0-adf7-2cc215a3bb1c`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:38:36,442::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:38:36,443::task::1185::TaskManager.Task::(prepare) Task=`36670388-67cf-48b0-adf7-2cc215a3bb1c`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:38:36,443::task::595::TaskManager.Task::(_updateState) Task=`36670388-67cf-48b0-adf7-2cc215a3bb1c`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:38:36,443::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:38:36,443::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:38:36,443::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:38:36,443::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:38:36,444::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:38:36,444::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:38:36,444::task::990::TaskManager.Task::(_decref) Task=`36670388-67cf-48b0-adf7-2cc215a3bb1c`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:38:37,325::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:37,413::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0447878 s, 22.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:38:38,006::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:38:38,648::task::595::TaskManager.Task::(_updateState) Task=`57053b07-81f1-40cb-b5ca-4a3799a49a30`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:38:38,648::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:38:38,648::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000728436', 'lastCheck': '6.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:38:38,648::task::1185::TaskManager.Task::(prepare) Task=`57053b07-81f1-40cb-b5ca-4a3799a49a30`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000728436', 'lastCheck': '6.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:38:38,649::task::595::TaskManager.Task::(_updateState) Task=`57053b07-81f1-40cb-b5ca-4a3799a49a30`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:38:38,649::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:38:38,649::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:38:38,649::task::990::TaskManager.Task::(_decref) Task=`57053b07-81f1-40cb-b5ca-4a3799a49a30`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:38:39,426::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:39,501::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0336496 s, 30.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:38:41,026::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:38:41,514::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:41,599::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381498 s, 26.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:38:42,050::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:38:42,060::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00235286 s, 1.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:38:43,617::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:43,692::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0337004 s, 30.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:38:44,045::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:38:45,705::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:45,782::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0325038 s, 31.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:38:46,462::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:38:46,462::task::595::TaskManager.Task::(_updateState) Task=`57871167-fa16-4021-ada7-8f4ef4c8eccb`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:38:46,463::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:38:46,463::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:38:46,463::task::1185::TaskManager.Task::(prepare) Task=`57871167-fa16-4021-ada7-8f4ef4c8eccb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:38:46,463::task::595::TaskManager.Task::(_updateState) Task=`57871167-fa16-4021-ada7-8f4ef4c8eccb`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:38:46,463::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:38:46,464::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:38:46,464::task::990::TaskManager.Task::(_decref) Task=`57871167-fa16-4021-ada7-8f4ef4c8eccb`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:38:46,468::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:38:46,468::task::595::TaskManager.Task::(_updateState) Task=`550b9a40-bb7e-43e2-a8dc-233edd1e05a0`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:38:46,469::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:38:46,469::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`afbc080c-3bfe-4f5b-9791-503757db55d1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:38:46,469::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:38:46,469::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:38:46,470::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`afbc080c-3bfe-4f5b-9791-503757db55d1`::Granted request >Thread-338::DEBUG::2014-05-13 12:38:46,470::task::827::TaskManager.Task::(resourceAcquired) Task=`550b9a40-bb7e-43e2-a8dc-233edd1e05a0`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:38:46,470::task::990::TaskManager.Task::(_decref) Task=`550b9a40-bb7e-43e2-a8dc-233edd1e05a0`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:38:46,471::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:38:46,471::task::1185::TaskManager.Task::(prepare) Task=`550b9a40-bb7e-43e2-a8dc-233edd1e05a0`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:38:46,471::task::595::TaskManager.Task::(_updateState) Task=`550b9a40-bb7e-43e2-a8dc-233edd1e05a0`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:38:46,472::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:38:46,472::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:38:46,472::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:38:46,472::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:38:46,472::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:38:46,473::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:38:46,473::task::990::TaskManager.Task::(_decref) Task=`550b9a40-bb7e-43e2-a8dc-233edd1e05a0`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:38:47,063::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:38:47,795::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:47,874::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361483 s, 28.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:38:49,887::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:49,961::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319913 s, 32.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:38:50,081::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:38:51,974::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 12:38:52,086::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:38:52,103::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000709938 s, 5.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:38:52,144::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0457571 s, 22.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:38:53,100::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:38:53,795::task::595::TaskManager.Task::(_updateState) Task=`c9577774-11ad-4186-8a1b-76492863e72f`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:38:53,795::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:38:53,795::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000709938', 'lastCheck': '1.7', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:38:53,795::task::1185::TaskManager.Task::(prepare) Task=`c9577774-11ad-4186-8a1b-76492863e72f`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000709938', 'lastCheck': '1.7', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:38:53,796::task::595::TaskManager.Task::(_updateState) Task=`c9577774-11ad-4186-8a1b-76492863e72f`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:38:53,796::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:38:53,796::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:38:53,796::task::990::TaskManager.Task::(_decref) Task=`c9577774-11ad-4186-8a1b-76492863e72f`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:38:54,158::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:54,235::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0333058 s, 30.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:38:56,121::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:38:56,247::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:56,328::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381216 s, 26.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:38:56,490::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [50b954f9] >Thread-66::DEBUG::2014-05-13 12:38:56,490::task::595::TaskManager.Task::(_updateState) Task=`f9009be6-e569-45ce-ad33-32d2f7d67e96`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:38:56,490::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:38:56,491::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:38:56,491::task::1185::TaskManager.Task::(prepare) Task=`f9009be6-e569-45ce-ad33-32d2f7d67e96`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:38:56,491::task::595::TaskManager.Task::(_updateState) Task=`f9009be6-e569-45ce-ad33-32d2f7d67e96`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:38:56,491::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:38:56,491::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:38:56,492::task::990::TaskManager.Task::(_decref) Task=`f9009be6-e569-45ce-ad33-32d2f7d67e96`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:38:56,497::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [50b954f9] >Thread-338::DEBUG::2014-05-13 12:38:56,497::task::595::TaskManager.Task::(_updateState) Task=`2d0a7bd2-4bf9-4222-b4ba-457eae21613e`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:38:56,498::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:38:56,498::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1c201d93-be1d-40d9-a4f4-fd83c091348b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:38:56,498::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:38:56,499::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:38:56,499::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1c201d93-be1d-40d9-a4f4-fd83c091348b`::Granted request >Thread-338::DEBUG::2014-05-13 12:38:56,499::task::827::TaskManager.Task::(resourceAcquired) Task=`2d0a7bd2-4bf9-4222-b4ba-457eae21613e`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:38:56,499::task::990::TaskManager.Task::(_decref) Task=`2d0a7bd2-4bf9-4222-b4ba-457eae21613e`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:38:56,500::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:38:56,500::task::1185::TaskManager.Task::(prepare) Task=`2d0a7bd2-4bf9-4222-b4ba-457eae21613e`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:38:56,500::task::595::TaskManager.Task::(_updateState) Task=`2d0a7bd2-4bf9-4222-b4ba-457eae21613e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:38:56,501::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:38:56,501::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:38:56,501::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:38:56,501::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:38:56,501::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:38:56,502::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:38:56,502::task::990::TaskManager.Task::(_decref) Task=`2d0a7bd2-4bf9-4222-b4ba-457eae21613e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:38:58,341::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:38:58,414::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318592 s, 32.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:38:59,139::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:39:00,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:39:00,503::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313181 s, 32.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:39:02,131::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:39:02,139::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000650701 s, 6.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:39:02,163::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:39:02,516::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:39:02,599::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381096 s, 26.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:39:04,612::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:39:04,688::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.03414 s, 30.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:39:05,182::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:39:06,521::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:39:06,522::task::595::TaskManager.Task::(_updateState) Task=`b639d1eb-a3f7-4825-a322-55573e25448a`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:39:06,522::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:39:06,522::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:39:06,523::task::1185::TaskManager.Task::(prepare) Task=`b639d1eb-a3f7-4825-a322-55573e25448a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:39:06,523::task::595::TaskManager.Task::(_updateState) Task=`b639d1eb-a3f7-4825-a322-55573e25448a`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:39:06,523::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:39:06,523::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:39:06,523::task::990::TaskManager.Task::(_decref) Task=`b639d1eb-a3f7-4825-a322-55573e25448a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:39:06,527::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:39:06,527::task::595::TaskManager.Task::(_updateState) Task=`e2265f81-c070-46a2-aada-b4a57d811037`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:39:06,528::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:39:06,528::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`463ed013-1c12-4b1d-9778-2195ae4e2e4e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:39:06,528::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:39:06,528::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:39:06,529::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`463ed013-1c12-4b1d-9778-2195ae4e2e4e`::Granted request >Thread-338::DEBUG::2014-05-13 12:39:06,529::task::827::TaskManager.Task::(resourceAcquired) Task=`e2265f81-c070-46a2-aada-b4a57d811037`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:39:06,529::task::990::TaskManager.Task::(_decref) Task=`e2265f81-c070-46a2-aada-b4a57d811037`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:39:06,530::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:39:06,530::task::1185::TaskManager.Task::(prepare) Task=`e2265f81-c070-46a2-aada-b4a57d811037`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:39:06,530::task::595::TaskManager.Task::(_updateState) Task=`e2265f81-c070-46a2-aada-b4a57d811037`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:39:06,530::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:39:06,530::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:39:06,531::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:39:06,531::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:39:06,531::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:39:06,531::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:39:06,531::task::990::TaskManager.Task::(_decref) Task=`e2265f81-c070-46a2-aada-b4a57d811037`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:39:06,702::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:39:07,060::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.31569 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:39:08,200::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:39:08,932::task::595::TaskManager.Task::(_updateState) Task=`97d609b8-feaf-4de5-9b90-5f875a627ba3`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:39:08,932::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:39:08,932::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000650701', 'lastCheck': '6.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:39:08,932::task::1185::TaskManager.Task::(prepare) Task=`97d609b8-feaf-4de5-9b90-5f875a627ba3`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000650701', 'lastCheck': '6.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:39:08,932::task::595::TaskManager.Task::(_updateState) Task=`97d609b8-feaf-4de5-9b90-5f875a627ba3`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:39:08,933::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:39:08,933::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:39:08,933::task::990::TaskManager.Task::(_decref) Task=`97d609b8-feaf-4de5-9b90-5f875a627ba3`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:39:09,073::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:39:09,155::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0395288 s, 25.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:39:11,168::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:39:11,220::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:39:11,253::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0391413 s, 26.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:39:12,146::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:39:12,154::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000574913 s, 7.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:39:13,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:39:13,341::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0334496 s, 30.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:39:14,237::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:39:15,354::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:39:15,436::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0394548 s, 26.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:39:16,548::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:39:16,549::task::595::TaskManager.Task::(_updateState) Task=`81ef529c-b667-44ca-a484-a27b9b86dc23`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:39:16,549::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:39:16,549::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:39:16,549::task::1185::TaskManager.Task::(prepare) Task=`81ef529c-b667-44ca-a484-a27b9b86dc23`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:39:16,550::task::595::TaskManager.Task::(_updateState) Task=`81ef529c-b667-44ca-a484-a27b9b86dc23`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:39:16,550::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:39:16,550::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:39:16,550::task::990::TaskManager.Task::(_decref) Task=`81ef529c-b667-44ca-a484-a27b9b86dc23`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:39:16,555::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:39:16,555::task::595::TaskManager.Task::(_updateState) Task=`ef98b811-ed85-4bc7-8c15-92fcd65e05fb`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:39:16,555::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:39:16,556::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f4078966-a031-4ba4-9add-51745b46dba7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:39:16,556::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:39:16,556::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:39:16,556::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f4078966-a031-4ba4-9add-51745b46dba7`::Granted request >Thread-338::DEBUG::2014-05-13 12:39:16,557::task::827::TaskManager.Task::(resourceAcquired) Task=`ef98b811-ed85-4bc7-8c15-92fcd65e05fb`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:39:16,557::task::990::TaskManager.Task::(_decref) Task=`ef98b811-ed85-4bc7-8c15-92fcd65e05fb`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:39:16,558::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:39:16,558::task::1185::TaskManager.Task::(prepare) Task=`ef98b811-ed85-4bc7-8c15-92fcd65e05fb`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:39:16,558::task::595::TaskManager.Task::(_updateState) Task=`ef98b811-ed85-4bc7-8c15-92fcd65e05fb`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:39:16,558::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:39:16,559::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:39:16,559::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:39:16,559::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:39:16,559::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:39:16,559::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:39:16,560::task::990::TaskManager.Task::(_decref) Task=`ef98b811-ed85-4bc7-8c15-92fcd65e05fb`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:39:17,256::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:39:17,449::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:39:17,528::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360277 s, 28.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:39:19,541::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:39:19,630::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0462665 s, 22.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:39:20,274::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:39:21,643::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:39:21,717::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306669 s, 33.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:39:22,161::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:39:22,170::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00163501 s, 2.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:39:23,292::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:39:23,730::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:39:23,816::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0442242 s, 23.2 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:39:24,066::task::595::TaskManager.Task::(_updateState) Task=`42f319ef-ba72-4613-a69f-6361093f9bfd`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:39:24,067::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:39:24,067::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00163501', 'lastCheck': '1.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:39:24,067::task::1185::TaskManager.Task::(prepare) Task=`42f319ef-ba72-4613-a69f-6361093f9bfd`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00163501', 'lastCheck': '1.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:39:24,067::task::595::TaskManager.Task::(_updateState) Task=`42f319ef-ba72-4613-a69f-6361093f9bfd`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:39:24,067::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:39:24,067::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:39:24,068::task::990::TaskManager.Task::(_decref) Task=`42f319ef-ba72-4613-a69f-6361093f9bfd`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:39:25,830::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:39:25,909::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363961 s, 28.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:39:26,309::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:39:26,579::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4a63f6c8] >Thread-66::DEBUG::2014-05-13 12:39:26,579::task::595::TaskManager.Task::(_updateState) Task=`abdfc54d-772a-4da4-b126-0282f29c9612`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:39:26,579::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:39:26,580::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:39:26,580::task::1185::TaskManager.Task::(prepare) Task=`abdfc54d-772a-4da4-b126-0282f29c9612`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:39:26,580::task::595::TaskManager.Task::(_updateState) Task=`abdfc54d-772a-4da4-b126-0282f29c9612`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:39:26,580::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:39:26,580::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:39:26,580::task::990::TaskManager.Task::(_decref) Task=`abdfc54d-772a-4da4-b126-0282f29c9612`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:39:26,585::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4a63f6c8] >Thread-338::DEBUG::2014-05-13 12:39:26,585::task::595::TaskManager.Task::(_updateState) Task=`27efa8bc-6d43-4347-afee-fb758709ed4b`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:39:26,585::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:39:26,586::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6b615f48-0583-4052-859c-08e70baaed75`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:39:26,586::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:39:26,586::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:39:26,587::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6b615f48-0583-4052-859c-08e70baaed75`::Granted request >Thread-338::DEBUG::2014-05-13 12:39:26,587::task::827::TaskManager.Task::(resourceAcquired) Task=`27efa8bc-6d43-4347-afee-fb758709ed4b`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:39:26,587::task::990::TaskManager.Task::(_decref) Task=`27efa8bc-6d43-4347-afee-fb758709ed4b`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:39:26,588::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:39:26,588::task::1185::TaskManager.Task::(prepare) Task=`27efa8bc-6d43-4347-afee-fb758709ed4b`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:39:26,588::task::595::TaskManager.Task::(_updateState) Task=`27efa8bc-6d43-4347-afee-fb758709ed4b`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:39:26,588::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:39:26,589::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:39:26,589::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:39:26,589::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:39:26,589::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:39:26,589::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:39:26,590::task::990::TaskManager.Task::(_decref) Task=`27efa8bc-6d43-4347-afee-fb758709ed4b`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:39:27,922::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:39:27,998::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0326805 s, 31.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:39:29,326::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:39:30,010::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:39:30,091::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384841 s, 26.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:39:32,104::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 12:39:32,196::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:39:32,214::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000613628 s, 6.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:39:32,268::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.118804 s, 8.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:39:32,344::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:39:34,281::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:39:34,355::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0327173 s, 31.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:39:35,363::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:39:36,368::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:39:36,442::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315998 s, 32.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:39:36,607::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3179350e] >Thread-66::DEBUG::2014-05-13 12:39:36,607::task::595::TaskManager.Task::(_updateState) Task=`0ec7f9dd-c72b-44c8-a470-cfa192a5906e`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:39:36,607::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:39:36,608::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:39:36,608::task::1185::TaskManager.Task::(prepare) Task=`0ec7f9dd-c72b-44c8-a470-cfa192a5906e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:39:36,608::task::595::TaskManager.Task::(_updateState) Task=`0ec7f9dd-c72b-44c8-a470-cfa192a5906e`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:39:36,608::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:39:36,608::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:39:36,609::task::990::TaskManager.Task::(_decref) Task=`0ec7f9dd-c72b-44c8-a470-cfa192a5906e`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:39:36,613::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3179350e] >Thread-338::DEBUG::2014-05-13 12:39:36,613::task::595::TaskManager.Task::(_updateState) Task=`3d730e15-7f4f-493a-bfd0-e5a1eb30521a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:39:36,614::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:39:36,614::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`dc9fca3e-99ce-4076-b44d-c77146ed7c68`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:39:36,614::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:39:36,615::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:39:36,615::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`dc9fca3e-99ce-4076-b44d-c77146ed7c68`::Granted request >Thread-338::DEBUG::2014-05-13 12:39:36,615::task::827::TaskManager.Task::(resourceAcquired) Task=`3d730e15-7f4f-493a-bfd0-e5a1eb30521a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:39:36,615::task::990::TaskManager.Task::(_decref) Task=`3d730e15-7f4f-493a-bfd0-e5a1eb30521a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:39:36,616::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:39:36,616::task::1185::TaskManager.Task::(prepare) Task=`3d730e15-7f4f-493a-bfd0-e5a1eb30521a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:39:36,616::task::595::TaskManager.Task::(_updateState) Task=`3d730e15-7f4f-493a-bfd0-e5a1eb30521a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:39:36,617::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:39:36,617::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:39:36,617::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:39:36,617::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:39:36,617::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:39:36,617::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:39:36,618::task::990::TaskManager.Task::(_decref) Task=`3d730e15-7f4f-493a-bfd0-e5a1eb30521a`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:39:38,382::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:39:38,455::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:39:38,534::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362368 s, 28.3 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:39:39,201::task::595::TaskManager.Task::(_updateState) Task=`8a60a6ee-4845-4185-803b-edde865b70c3`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:39:39,201::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:39:39,202::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000613628', 'lastCheck': '7.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:39:39,202::task::1185::TaskManager.Task::(prepare) Task=`8a60a6ee-4845-4185-803b-edde865b70c3`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000613628', 'lastCheck': '7.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:39:39,202::task::595::TaskManager.Task::(_updateState) Task=`8a60a6ee-4845-4185-803b-edde865b70c3`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:39:39,202::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:39:39,202::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:39:39,203::task::990::TaskManager.Task::(_decref) Task=`8a60a6ee-4845-4185-803b-edde865b70c3`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:39:40,547::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:39:40,627::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387455 s, 26.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:39:41,402::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:39:42,234::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:39:42,242::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000651012 s, 6.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:39:42,640::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:39:42,721::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0380707 s, 26.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:39:44,420::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:39:44,734::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:39:44,815::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0386105 s, 26.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:39:46,635::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:39:46,636::task::595::TaskManager.Task::(_updateState) Task=`9c3115d6-73b8-41ad-ac4d-39ade26dab5f`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:39:46,636::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:39:46,637::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:39:46,637::task::1185::TaskManager.Task::(prepare) Task=`9c3115d6-73b8-41ad-ac4d-39ade26dab5f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:39:46,637::task::595::TaskManager.Task::(_updateState) Task=`9c3115d6-73b8-41ad-ac4d-39ade26dab5f`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:39:46,637::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:39:46,637::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:39:46,637::task::990::TaskManager.Task::(_decref) Task=`9c3115d6-73b8-41ad-ac4d-39ade26dab5f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:39:46,642::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:39:46,642::task::595::TaskManager.Task::(_updateState) Task=`a7483317-5580-401e-b527-adb169801e98`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:39:46,643::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:39:46,643::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`cee7fc23-fcae-4e4f-adcf-76c87b117b75`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:39:46,643::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:39:46,644::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:39:46,644::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`cee7fc23-fcae-4e4f-adcf-76c87b117b75`::Granted request >Thread-338::DEBUG::2014-05-13 12:39:46,644::task::827::TaskManager.Task::(resourceAcquired) Task=`a7483317-5580-401e-b527-adb169801e98`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:39:46,644::task::990::TaskManager.Task::(_decref) Task=`a7483317-5580-401e-b527-adb169801e98`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:39:46,645::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:39:46,645::task::1185::TaskManager.Task::(prepare) Task=`a7483317-5580-401e-b527-adb169801e98`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:39:46,645::task::595::TaskManager.Task::(_updateState) Task=`a7483317-5580-401e-b527-adb169801e98`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:39:46,646::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:39:46,646::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:39:46,646::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:39:46,646::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:39:46,646::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:39:46,646::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:39:46,647::task::990::TaskManager.Task::(_decref) Task=`a7483317-5580-401e-b527-adb169801e98`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:39:46,828::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:39:46,907::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383234 s, 26.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:39:47,438::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:39:48,920::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:39:49,286::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.321107 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:39:50,458::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:39:51,300::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:39:51,376::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0332969 s, 30.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:39:52,249::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:39:52,257::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000635689 s, 6.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:39:53,390::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:39:53,469::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363072 s, 28.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:39:53,480::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:39:54,342::task::595::TaskManager.Task::(_updateState) Task=`8ee9110d-2c53-4f4e-b7e8-a8dc874626e8`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:39:54,342::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:39:54,342::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000635689', 'lastCheck': '2.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:39:54,342::task::1185::TaskManager.Task::(prepare) Task=`8ee9110d-2c53-4f4e-b7e8-a8dc874626e8`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000635689', 'lastCheck': '2.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:39:54,342::task::595::TaskManager.Task::(_updateState) Task=`8ee9110d-2c53-4f4e-b7e8-a8dc874626e8`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:39:54,343::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:39:54,343::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:39:54,343::task::990::TaskManager.Task::(_decref) Task=`8ee9110d-2c53-4f4e-b7e8-a8dc874626e8`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:39:55,484::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:39:55,566::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383591 s, 26.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:39:56,497::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:39:56,667::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [303c64a0] >Thread-66::DEBUG::2014-05-13 12:39:56,668::task::595::TaskManager.Task::(_updateState) Task=`30e09926-44b4-4eff-b93c-c8abde27ea16`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:39:56,668::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:39:56,668::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:39:56,668::task::1185::TaskManager.Task::(prepare) Task=`30e09926-44b4-4eff-b93c-c8abde27ea16`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:39:56,669::task::595::TaskManager.Task::(_updateState) Task=`30e09926-44b4-4eff-b93c-c8abde27ea16`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:39:56,669::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:39:56,669::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:39:56,669::task::990::TaskManager.Task::(_decref) Task=`30e09926-44b4-4eff-b93c-c8abde27ea16`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:39:56,673::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [303c64a0] >Thread-338::DEBUG::2014-05-13 12:39:56,674::task::595::TaskManager.Task::(_updateState) Task=`b1fddafd-8a36-4a2a-87a1-204819fc19a5`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:39:56,674::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:39:56,674::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`cdd370a4-6ed8-4428-8c99-ef411d17d736`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:39:56,674::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:39:56,675::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:39:56,675::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`cdd370a4-6ed8-4428-8c99-ef411d17d736`::Granted request >Thread-338::DEBUG::2014-05-13 12:39:56,675::task::827::TaskManager.Task::(resourceAcquired) Task=`b1fddafd-8a36-4a2a-87a1-204819fc19a5`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:39:56,675::task::990::TaskManager.Task::(_decref) Task=`b1fddafd-8a36-4a2a-87a1-204819fc19a5`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:39:56,676::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:39:56,676::task::1185::TaskManager.Task::(prepare) Task=`b1fddafd-8a36-4a2a-87a1-204819fc19a5`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:39:56,677::task::595::TaskManager.Task::(_updateState) Task=`b1fddafd-8a36-4a2a-87a1-204819fc19a5`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:39:56,677::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:39:56,677::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:39:56,677::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:39:56,677::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:39:56,678::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:39:56,678::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:39:56,678::task::990::TaskManager.Task::(_decref) Task=`b1fddafd-8a36-4a2a-87a1-204819fc19a5`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:39:57,579::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:39:57,655::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0337999 s, 30.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:39:59,516::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:39:59,668::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:00,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.365662 s, 2.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:40:02,090::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:02,172::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387657 s, 26.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:40:02,289::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:40:02,298::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00126645 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:40:02,536::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:40:04,185::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:04,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0332477 s, 30.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:40:05,555::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:40:06,292::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:06,366::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310397 s, 33.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:40:06,698::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [dc1f215] >Thread-66::DEBUG::2014-05-13 12:40:06,698::task::595::TaskManager.Task::(_updateState) Task=`f2e4cb37-d892-4c47-ae20-7c6c7aee77a4`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:40:06,699::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:40:06,699::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:40:06,699::task::1185::TaskManager.Task::(prepare) Task=`f2e4cb37-d892-4c47-ae20-7c6c7aee77a4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:40:06,699::task::595::TaskManager.Task::(_updateState) Task=`f2e4cb37-d892-4c47-ae20-7c6c7aee77a4`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:40:06,699::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:40:06,700::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:40:06,700::task::990::TaskManager.Task::(_decref) Task=`f2e4cb37-d892-4c47-ae20-7c6c7aee77a4`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:40:06,704::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [dc1f215] >Thread-338::DEBUG::2014-05-13 12:40:06,704::task::595::TaskManager.Task::(_updateState) Task=`891b41a4-b610-498a-9e1d-e6aa3029afe8`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:40:06,705::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:40:06,705::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b5ddb5cf-d091-48b9-962b-27ca149588ff`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:40:06,705::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:40:06,706::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:40:06,706::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b5ddb5cf-d091-48b9-962b-27ca149588ff`::Granted request >Thread-338::DEBUG::2014-05-13 12:40:06,706::task::827::TaskManager.Task::(resourceAcquired) Task=`891b41a4-b610-498a-9e1d-e6aa3029afe8`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:40:06,706::task::990::TaskManager.Task::(_decref) Task=`891b41a4-b610-498a-9e1d-e6aa3029afe8`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:40:06,707::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:40:06,707::task::1185::TaskManager.Task::(prepare) Task=`891b41a4-b610-498a-9e1d-e6aa3029afe8`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:40:06,707::task::595::TaskManager.Task::(_updateState) Task=`891b41a4-b610-498a-9e1d-e6aa3029afe8`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:40:06,708::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:40:06,708::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:40:06,708::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:40:06,708::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:40:06,708::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:40:06,709::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:40:06,709::task::990::TaskManager.Task::(_decref) Task=`891b41a4-b610-498a-9e1d-e6aa3029afe8`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:40:08,379::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:08,458::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356806 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:40:08,579::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:40:09,488::task::595::TaskManager.Task::(_updateState) Task=`ef194a10-a28f-4125-a306-c73817b86b50`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:40:09,489::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:40:09,489::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00126645', 'lastCheck': '7.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:40:09,489::task::1185::TaskManager.Task::(prepare) Task=`ef194a10-a28f-4125-a306-c73817b86b50`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00126645', 'lastCheck': '7.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:40:09,489::task::595::TaskManager.Task::(_updateState) Task=`ef194a10-a28f-4125-a306-c73817b86b50`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:40:09,489::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:40:09,489::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:40:09,490::task::990::TaskManager.Task::(_decref) Task=`ef194a10-a28f-4125-a306-c73817b86b50`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:40:10,471::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:10,554::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0380276 s, 26.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:40:11,602::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:40:12,302::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 41830dad-f012-4b4d-b636-863f66d5ca4b >Thread-322::DEBUG::2014-05-13 12:40:12,303::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-322::DEBUG::2014-05-13 12:40:12,305::blockSD::336::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:40:12,314::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.0021679 s, 945 kB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:40:12,315::misc::287::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.0021679 s, 945 kB/s'], size: 2048 >Thread-322::DEBUG::2014-05-13 12:40:12,315::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-322::WARNING::2014-05-13 12:40:12,315::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-322::DEBUG::2014-05-13 12:40:12,315::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-322::DEBUG::2014-05-13 12:40:12,316::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:40:12,316::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:40:12,316::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:40:12,317::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:40:12,317::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:40:12,318::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:12,569::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:12,661::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0492439 s, 20.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:40:12,896::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:40:12,898::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:40:12,898::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'LOCKPOLICY=', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', 'DESCRIPTION=ISCSI', 'LOGBLKSIZE=512', 'TYPE=ISCSI', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'PHYBLKSIZE=512', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', 'CLASS=Data', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'POOL_DESCRIPTION=DC_2', 'POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'MASTER_VERSION=1', 'ROLE=Master', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', '_SHA_CKSUM=28bb2d7cc60e0d779eb69fe93d050c16c769f069', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0'] >Thread-322::INFO::2014-05-13 12:40:12,899::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_imageNS already registered >Thread-322::INFO::2014-05-13 12:40:12,900::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_volumeNS already registered >Thread-322::INFO::2014-05-13 12:40:12,900::blockSD::452::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_lvmActivationNS already registered >Thread-322::DEBUG::2014-05-13 12:40:12,901::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 12:40:13,069::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:40:13,073::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:40:13,081::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000691597 s, 5.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:40:14,622::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:40:14,674::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:14,781::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.048984 s, 20.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:40:16,727::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:40:16,727::task::595::TaskManager.Task::(_updateState) Task=`52a55a27-01dc-4750-8969-6a04a2da2239`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:40:16,728::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:40:16,728::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:40:16,728::task::1185::TaskManager.Task::(prepare) Task=`52a55a27-01dc-4750-8969-6a04a2da2239`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:40:16,728::task::595::TaskManager.Task::(_updateState) Task=`52a55a27-01dc-4750-8969-6a04a2da2239`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:40:16,728::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:40:16,729::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:40:16,729::task::990::TaskManager.Task::(_decref) Task=`52a55a27-01dc-4750-8969-6a04a2da2239`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:40:16,733::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:40:16,733::task::595::TaskManager.Task::(_updateState) Task=`7be9b195-183c-44c4-a997-0b5dd12bcceb`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:40:16,734::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:40:16,734::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2414fb88-b9f7-46e3-be5f-839be6661aca`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:40:16,734::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:40:16,735::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:40:16,735::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`2414fb88-b9f7-46e3-be5f-839be6661aca`::Granted request >Thread-338::DEBUG::2014-05-13 12:40:16,735::task::827::TaskManager.Task::(resourceAcquired) Task=`7be9b195-183c-44c4-a997-0b5dd12bcceb`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:40:16,735::task::990::TaskManager.Task::(_decref) Task=`7be9b195-183c-44c4-a997-0b5dd12bcceb`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:40:16,736::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:40:16,736::task::1185::TaskManager.Task::(prepare) Task=`7be9b195-183c-44c4-a997-0b5dd12bcceb`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:40:16,736::task::595::TaskManager.Task::(_updateState) Task=`7be9b195-183c-44c4-a997-0b5dd12bcceb`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:40:16,737::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:40:16,737::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:40:16,737::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:40:16,737::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:40:16,737::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:40:16,737::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:40:16,738::task::990::TaskManager.Task::(_decref) Task=`7be9b195-183c-44c4-a997-0b5dd12bcceb`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:40:16,799::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:16,884::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0396741 s, 25.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:40:17,640::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:40:18,897::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:18,971::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309981 s, 33.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:40:20,658::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:40:20,984::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:21,062::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355611 s, 28.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:40:23,075::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 12:40:23,089::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:40:23,105::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000661965 s, 6.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:40:23,188::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0716343 s, 14.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:40:23,675::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:40:24,624::task::595::TaskManager.Task::(_updateState) Task=`2370c956-8431-4920-809f-29b00445cf3e`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:40:24,625::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:40:24,625::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000661965', 'lastCheck': '1.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:40:24,625::task::1185::TaskManager.Task::(prepare) Task=`2370c956-8431-4920-809f-29b00445cf3e`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000661965', 'lastCheck': '1.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:40:24,625::task::595::TaskManager.Task::(_updateState) Task=`2370c956-8431-4920-809f-29b00445cf3e`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:40:24,625::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:40:24,625::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:40:24,626::task::990::TaskManager.Task::(_decref) Task=`2370c956-8431-4920-809f-29b00445cf3e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:40:25,201::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:25,276::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0337617 s, 30.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:40:26,702::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:40:26,757::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:40:26,758::task::595::TaskManager.Task::(_updateState) Task=`3658207c-b8ee-4fb8-8e39-e6354b9b36d4`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:40:26,758::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:40:26,758::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:40:26,759::task::1185::TaskManager.Task::(prepare) Task=`3658207c-b8ee-4fb8-8e39-e6354b9b36d4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:40:26,759::task::595::TaskManager.Task::(_updateState) Task=`3658207c-b8ee-4fb8-8e39-e6354b9b36d4`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:40:26,759::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:40:26,759::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:40:26,759::task::990::TaskManager.Task::(_decref) Task=`3658207c-b8ee-4fb8-8e39-e6354b9b36d4`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:40:26,764::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:40:26,764::task::595::TaskManager.Task::(_updateState) Task=`18b8b124-0c57-4bfa-9987-188352e6dc63`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:40:26,764::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:40:26,765::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`63dada7e-feed-48b2-91fc-8eb1a6cb1ffc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:40:26,765::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:40:26,765::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:40:26,766::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`63dada7e-feed-48b2-91fc-8eb1a6cb1ffc`::Granted request >Thread-338::DEBUG::2014-05-13 12:40:26,766::task::827::TaskManager.Task::(resourceAcquired) Task=`18b8b124-0c57-4bfa-9987-188352e6dc63`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:40:26,766::task::990::TaskManager.Task::(_decref) Task=`18b8b124-0c57-4bfa-9987-188352e6dc63`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:40:26,767::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:40:26,767::task::1185::TaskManager.Task::(prepare) Task=`18b8b124-0c57-4bfa-9987-188352e6dc63`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:40:26,767::task::595::TaskManager.Task::(_updateState) Task=`18b8b124-0c57-4bfa-9987-188352e6dc63`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:40:26,767::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:40:26,768::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:40:26,768::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:40:26,768::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:40:26,768::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:40:26,768::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:40:26,769::task::990::TaskManager.Task::(_decref) Task=`18b8b124-0c57-4bfa-9987-188352e6dc63`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:40:27,289::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:27,376::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0436657 s, 23.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:40:29,389::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:29,469::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037999 s, 26.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:40:29,719::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:40:31,482::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:31,562::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0389042 s, 26.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:40:32,738::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:40:33,131::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:40:33,139::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000800835 s, 5.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:40:33,575::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:33,650::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0332054 s, 30.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:40:35,663::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:35,743::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0354874 s, 28.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:40:35,758::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:40:36,787::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [28fd42b9] >Thread-66::DEBUG::2014-05-13 12:40:36,788::task::595::TaskManager.Task::(_updateState) Task=`c5d82f89-5e71-46e3-b000-37298daf8cca`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:40:36,788::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:40:36,789::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:40:36,789::task::1185::TaskManager.Task::(prepare) Task=`c5d82f89-5e71-46e3-b000-37298daf8cca`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:40:36,789::task::595::TaskManager.Task::(_updateState) Task=`c5d82f89-5e71-46e3-b000-37298daf8cca`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:40:36,789::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:40:36,789::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:40:36,789::task::990::TaskManager.Task::(_decref) Task=`c5d82f89-5e71-46e3-b000-37298daf8cca`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:40:36,794::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [28fd42b9] >Thread-338::DEBUG::2014-05-13 12:40:36,794::task::595::TaskManager.Task::(_updateState) Task=`2ff5b92d-b204-4226-a7e3-00345ce95e92`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:40:36,794::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:40:36,794::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`68c4052d-e5ad-48ae-9aca-fc4e63529aad`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:40:36,795::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:40:36,795::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:40:36,795::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`68c4052d-e5ad-48ae-9aca-fc4e63529aad`::Granted request >Thread-338::DEBUG::2014-05-13 12:40:36,795::task::827::TaskManager.Task::(resourceAcquired) Task=`2ff5b92d-b204-4226-a7e3-00345ce95e92`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:40:36,795::task::990::TaskManager.Task::(_decref) Task=`2ff5b92d-b204-4226-a7e3-00345ce95e92`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:40:36,796::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:40:36,796::task::1185::TaskManager.Task::(prepare) Task=`2ff5b92d-b204-4226-a7e3-00345ce95e92`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:40:36,796::task::595::TaskManager.Task::(_updateState) Task=`2ff5b92d-b204-4226-a7e3-00345ce95e92`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:40:36,797::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:40:36,797::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:40:36,797::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:40:36,797::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:40:36,797::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:40:36,797::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:40:36,798::task::990::TaskManager.Task::(_decref) Task=`2ff5b92d-b204-4226-a7e3-00345ce95e92`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:40:37,755::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:37,830::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315191 s, 32.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:40:38,778::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:40:39,755::task::595::TaskManager.Task::(_updateState) Task=`6770e782-e4cc-4660-aee8-e8f12d5bd2af`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:40:39,755::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:40:39,756::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000800835', 'lastCheck': '6.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:40:39,756::task::1185::TaskManager.Task::(prepare) Task=`6770e782-e4cc-4660-aee8-e8f12d5bd2af`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000800835', 'lastCheck': '6.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:40:39,756::task::595::TaskManager.Task::(_updateState) Task=`6770e782-e4cc-4660-aee8-e8f12d5bd2af`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:40:39,756::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:40:39,756::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:40:39,756::task::990::TaskManager.Task::(_decref) Task=`6770e782-e4cc-4660-aee8-e8f12d5bd2af`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:40:39,843::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:39,931::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0457675 s, 22.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:40:41,796::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:40:41,945::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:42,023::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353385 s, 29.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:40:43,146::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:40:43,154::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000690164 s, 5.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:40:44,036::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:44,115::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362659 s, 28.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:40:44,816::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:40:46,128::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:46,211::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0396912 s, 25.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:40:46,815::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:40:46,815::task::595::TaskManager.Task::(_updateState) Task=`5e6ab374-a9ab-4023-879d-fa5268ae3d3f`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:40:46,815::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:40:46,816::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:40:46,816::task::1185::TaskManager.Task::(prepare) Task=`5e6ab374-a9ab-4023-879d-fa5268ae3d3f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:40:46,816::task::595::TaskManager.Task::(_updateState) Task=`5e6ab374-a9ab-4023-879d-fa5268ae3d3f`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:40:46,816::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:40:46,816::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:40:46,817::task::990::TaskManager.Task::(_decref) Task=`5e6ab374-a9ab-4023-879d-fa5268ae3d3f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:40:46,821::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:40:46,822::task::595::TaskManager.Task::(_updateState) Task=`be3f4223-52c4-4453-b0bd-4b63508a333a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:40:46,822::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:40:46,822::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b57a17a2-f3fa-48aa-884d-4354524a9ae5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:40:46,823::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:40:46,823::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:40:46,823::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b57a17a2-f3fa-48aa-884d-4354524a9ae5`::Granted request >Thread-338::DEBUG::2014-05-13 12:40:46,823::task::827::TaskManager.Task::(resourceAcquired) Task=`be3f4223-52c4-4453-b0bd-4b63508a333a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:40:46,824::task::990::TaskManager.Task::(_decref) Task=`be3f4223-52c4-4453-b0bd-4b63508a333a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:40:46,824::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:40:46,825::task::1185::TaskManager.Task::(prepare) Task=`be3f4223-52c4-4453-b0bd-4b63508a333a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:40:46,825::task::595::TaskManager.Task::(_updateState) Task=`be3f4223-52c4-4453-b0bd-4b63508a333a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:40:46,825::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:40:46,825::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:40:46,825::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:40:46,826::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:40:46,826::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:40:46,826::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:40:46,826::task::990::TaskManager.Task::(_decref) Task=`be3f4223-52c4-4453-b0bd-4b63508a333a`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:40:47,835::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:40:48,224::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:48,306::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0386564 s, 26.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:40:50,314::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:50,409::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0318623 s, 32.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:40:50,853::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:40:52,422::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:52,829::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.365113 s, 2.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:40:53,162::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:40:53,170::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000614833 s, 6.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:40:53,871::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:40:54,842::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 12:40:54,882::task::595::TaskManager.Task::(_updateState) Task=`c08ff8dc-450b-4ec1-af87-65b40f38f655`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:40:54,882::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:40:54,882::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000614833', 'lastCheck': '1.7', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:40:54,883::task::1185::TaskManager.Task::(prepare) Task=`c08ff8dc-450b-4ec1-af87-65b40f38f655`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000614833', 'lastCheck': '1.7', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:40:54,883::task::595::TaskManager.Task::(_updateState) Task=`c08ff8dc-450b-4ec1-af87-65b40f38f655`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:40:54,883::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:40:54,883::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:40:54,883::task::990::TaskManager.Task::(_decref) Task=`c08ff8dc-450b-4ec1-af87-65b40f38f655`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:40:54,954::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0697022 s, 14.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:40:56,847::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:40:56,847::task::595::TaskManager.Task::(_updateState) Task=`762f25d3-d1ff-4ea0-a71d-a34678fe64ca`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:40:56,847::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:40:56,848::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:40:56,848::task::1185::TaskManager.Task::(prepare) Task=`762f25d3-d1ff-4ea0-a71d-a34678fe64ca`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:40:56,848::task::595::TaskManager.Task::(_updateState) Task=`762f25d3-d1ff-4ea0-a71d-a34678fe64ca`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:40:56,848::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:40:56,849::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:40:56,849::task::990::TaskManager.Task::(_decref) Task=`762f25d3-d1ff-4ea0-a71d-a34678fe64ca`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:40:56,853::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:40:56,854::task::595::TaskManager.Task::(_updateState) Task=`fefe7131-2955-4575-97c5-f914ba3fd938`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:40:56,854::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:40:56,854::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0cac6443-d0bb-42d6-954e-2a194f1ac5dd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:40:56,855::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:40:56,855::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:40:56,855::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0cac6443-d0bb-42d6-954e-2a194f1ac5dd`::Granted request >Thread-338::DEBUG::2014-05-13 12:40:56,856::task::827::TaskManager.Task::(resourceAcquired) Task=`fefe7131-2955-4575-97c5-f914ba3fd938`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:40:56,856::task::990::TaskManager.Task::(_decref) Task=`fefe7131-2955-4575-97c5-f914ba3fd938`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:40:56,856::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:40:56,857::task::1185::TaskManager.Task::(prepare) Task=`fefe7131-2955-4575-97c5-f914ba3fd938`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:40:56,857::task::595::TaskManager.Task::(_updateState) Task=`fefe7131-2955-4575-97c5-f914ba3fd938`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:40:56,857::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:40:56,857::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:40:56,858::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:40:56,858::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:40:56,858::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:40:56,858::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:40:56,858::task::990::TaskManager.Task::(_decref) Task=`fefe7131-2955-4575-97c5-f914ba3fd938`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:40:56,889::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:40:56,966::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:57,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0367282 s, 27.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:40:59,061::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:40:59,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306811 s, 33.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:40:59,910::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:41:01,147::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:01,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0367481 s, 27.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:41:02,928::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:41:03,177::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:41:03,185::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00068811 s, 6.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:41:03,241::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:03,319::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0354462 s, 28.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:41:05,328::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:05,776::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.405177 s, 2.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:41:05,946::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:41:06,876::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3256e43f] >Thread-66::DEBUG::2014-05-13 12:41:06,876::task::595::TaskManager.Task::(_updateState) Task=`3b1bca01-dfc8-4333-a1a1-73a8089333b1`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:41:06,876::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:41:06,877::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:41:06,877::task::1185::TaskManager.Task::(prepare) Task=`3b1bca01-dfc8-4333-a1a1-73a8089333b1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:41:06,877::task::595::TaskManager.Task::(_updateState) Task=`3b1bca01-dfc8-4333-a1a1-73a8089333b1`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:41:06,877::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:41:06,877::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:41:06,877::task::990::TaskManager.Task::(_decref) Task=`3b1bca01-dfc8-4333-a1a1-73a8089333b1`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:41:06,883::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3256e43f] >Thread-338::DEBUG::2014-05-13 12:41:06,883::task::595::TaskManager.Task::(_updateState) Task=`548a0cd8-76e0-4145-9ad5-52459a69043a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:41:06,883::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:41:06,883::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`469cd7f6-7775-4059-8c81-5522a5a4ff4d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:41:06,884::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:41:06,884::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:41:06,884::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`469cd7f6-7775-4059-8c81-5522a5a4ff4d`::Granted request >Thread-338::DEBUG::2014-05-13 12:41:06,884::task::827::TaskManager.Task::(resourceAcquired) Task=`548a0cd8-76e0-4145-9ad5-52459a69043a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:41:06,884::task::990::TaskManager.Task::(_decref) Task=`548a0cd8-76e0-4145-9ad5-52459a69043a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:41:06,885::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:41:06,885::task::1185::TaskManager.Task::(prepare) Task=`548a0cd8-76e0-4145-9ad5-52459a69043a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:41:06,885::task::595::TaskManager.Task::(_updateState) Task=`548a0cd8-76e0-4145-9ad5-52459a69043a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:41:06,886::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:41:06,886::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:41:06,886::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:41:06,886::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:41:06,886::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:41:06,886::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:41:06,887::task::990::TaskManager.Task::(_decref) Task=`548a0cd8-76e0-4145-9ad5-52459a69043a`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:41:07,789::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:07,862::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309126 s, 33.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:41:08,964::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:41:09,875::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:09,952::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0326622 s, 31.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:41:10,012::task::595::TaskManager.Task::(_updateState) Task=`b6302c6a-3a08-40c4-9d0c-a1611a98ce57`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:41:10,012::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:41:10,012::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00068811', 'lastCheck': '6.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:41:10,012::task::1185::TaskManager.Task::(prepare) Task=`b6302c6a-3a08-40c4-9d0c-a1611a98ce57`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00068811', 'lastCheck': '6.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:41:10,013::task::595::TaskManager.Task::(_updateState) Task=`b6302c6a-3a08-40c4-9d0c-a1611a98ce57`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:41:10,013::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:41:10,013::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:41:10,013::task::990::TaskManager.Task::(_decref) Task=`b6302c6a-3a08-40c4-9d0c-a1611a98ce57`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:41:11,965::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:41:11,984::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:41:12,050::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.03991 s, 25.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:41:13,192::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:41:13,200::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00065537 s, 6.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:41:14,063::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:14,145::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383894 s, 26.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:41:15,001::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:41:16,158::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:16,248::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0474093 s, 21.6 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:41:16,907::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:41:16,907::task::595::TaskManager.Task::(_updateState) Task=`8a050bb9-0fa4-4af8-b36b-7e4423c9bf4d`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:41:16,907::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:41:16,908::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:41:16,908::task::1185::TaskManager.Task::(prepare) Task=`8a050bb9-0fa4-4af8-b36b-7e4423c9bf4d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:41:16,908::task::595::TaskManager.Task::(_updateState) Task=`8a050bb9-0fa4-4af8-b36b-7e4423c9bf4d`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:41:16,908::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:41:16,908::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:41:16,909::task::990::TaskManager.Task::(_decref) Task=`8a050bb9-0fa4-4af8-b36b-7e4423c9bf4d`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:41:16,914::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:41:16,915::task::595::TaskManager.Task::(_updateState) Task=`23bff94e-2fd1-4531-b8ab-79a77c174235`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:41:16,915::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:41:16,915::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c861698d-fd6d-4124-918c-9d30dc58e8b0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:41:16,915::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:41:16,916::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:41:16,916::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c861698d-fd6d-4124-918c-9d30dc58e8b0`::Granted request >Thread-338::DEBUG::2014-05-13 12:41:16,916::task::827::TaskManager.Task::(resourceAcquired) Task=`23bff94e-2fd1-4531-b8ab-79a77c174235`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:41:16,916::task::990::TaskManager.Task::(_decref) Task=`23bff94e-2fd1-4531-b8ab-79a77c174235`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:41:16,917::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:41:16,917::task::1185::TaskManager.Task::(prepare) Task=`23bff94e-2fd1-4531-b8ab-79a77c174235`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:41:16,918::task::595::TaskManager.Task::(_updateState) Task=`23bff94e-2fd1-4531-b8ab-79a77c174235`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:41:16,918::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:41:16,918::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:41:16,918::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:41:16,918::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:41:16,919::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:41:16,919::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:41:16,919::task::990::TaskManager.Task::(_decref) Task=`23bff94e-2fd1-4531-b8ab-79a77c174235`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:41:18,023::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:41:18,261::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:18,341::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0377794 s, 27.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:41:20,349::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:20,426::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323147 s, 31.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:41:21,041::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:41:22,438::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:22,521::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0396854 s, 25.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:41:23,207::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:41:23,215::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000629291 s, 6.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:41:24,060::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:41:24,534::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:24,610::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0331594 s, 30.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:41:25,144::task::595::TaskManager.Task::(_updateState) Task=`940e27e7-c0f8-4e3d-8055-59b75a570acb`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:41:25,144::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:41:25,145::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000629291', 'lastCheck': '1.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:41:25,145::task::1185::TaskManager.Task::(prepare) Task=`940e27e7-c0f8-4e3d-8055-59b75a570acb`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000629291', 'lastCheck': '1.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:41:25,145::task::595::TaskManager.Task::(_updateState) Task=`940e27e7-c0f8-4e3d-8055-59b75a570acb`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:41:25,145::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:41:25,145::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:41:25,145::task::990::TaskManager.Task::(_decref) Task=`940e27e7-c0f8-4e3d-8055-59b75a570acb`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:41:26,623::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:26,704::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.039207 s, 26.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:41:26,940::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6019f20] >Thread-66::DEBUG::2014-05-13 12:41:26,940::task::595::TaskManager.Task::(_updateState) Task=`2b378eb9-dfe8-4f96-a3ed-5fa5b50e2102`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:41:26,941::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:41:26,941::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:41:26,941::task::1185::TaskManager.Task::(prepare) Task=`2b378eb9-dfe8-4f96-a3ed-5fa5b50e2102`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:41:26,941::task::595::TaskManager.Task::(_updateState) Task=`2b378eb9-dfe8-4f96-a3ed-5fa5b50e2102`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:41:26,942::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:41:26,942::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:41:26,942::task::990::TaskManager.Task::(_decref) Task=`2b378eb9-dfe8-4f96-a3ed-5fa5b50e2102`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:41:26,947::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6019f20] >Thread-338::DEBUG::2014-05-13 12:41:26,947::task::595::TaskManager.Task::(_updateState) Task=`23a1282c-08da-4b21-babb-1ff6db6a9fb0`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:41:26,947::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:41:26,948::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`dd0197fd-77d0-486a-83d2-271084bac57a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:41:26,948::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:41:26,948::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:41:26,948::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`dd0197fd-77d0-486a-83d2-271084bac57a`::Granted request >Thread-338::DEBUG::2014-05-13 12:41:26,949::task::827::TaskManager.Task::(resourceAcquired) Task=`23a1282c-08da-4b21-babb-1ff6db6a9fb0`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:41:26,949::task::990::TaskManager.Task::(_decref) Task=`23a1282c-08da-4b21-babb-1ff6db6a9fb0`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:41:26,950::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:41:26,950::task::1185::TaskManager.Task::(prepare) Task=`23a1282c-08da-4b21-babb-1ff6db6a9fb0`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:41:26,950::task::595::TaskManager.Task::(_updateState) Task=`23a1282c-08da-4b21-babb-1ff6db6a9fb0`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:41:26,950::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:41:26,951::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:41:26,951::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:41:26,951::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:41:26,951::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:41:26,951::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:41:26,952::task::990::TaskManager.Task::(_decref) Task=`23a1282c-08da-4b21-babb-1ff6db6a9fb0`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:41:27,078::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:41:28,717::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:28,791::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0304381 s, 33.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:41:30,096::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:41:30,804::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:30,899::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0521119 s, 19.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:41:32,912::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:32,992::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0380414 s, 26.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:41:33,115::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:41:33,222::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:41:33,230::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000734718 s, 5.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:41:35,005::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:35,080::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0330958 s, 30.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:41:36,133::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:41:36,972::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [72471f8f] >Thread-66::DEBUG::2014-05-13 12:41:36,973::task::595::TaskManager.Task::(_updateState) Task=`d893bac7-5888-46e8-801b-6fd5608c9fd2`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:41:36,973::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:41:36,973::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:41:36,973::task::1185::TaskManager.Task::(prepare) Task=`d893bac7-5888-46e8-801b-6fd5608c9fd2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:41:36,973::task::595::TaskManager.Task::(_updateState) Task=`d893bac7-5888-46e8-801b-6fd5608c9fd2`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:41:36,973::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:41:36,974::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:41:36,974::task::990::TaskManager.Task::(_decref) Task=`d893bac7-5888-46e8-801b-6fd5608c9fd2`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:41:36,979::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [72471f8f] >Thread-338::DEBUG::2014-05-13 12:41:36,980::task::595::TaskManager.Task::(_updateState) Task=`9b25966c-0081-4a9d-acbd-6cc9f4bf7a54`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:41:36,980::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:41:36,980::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6677ca9a-ff91-473e-866c-8d07dfd37d36`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:41:36,980::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:41:36,981::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:41:36,981::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6677ca9a-ff91-473e-866c-8d07dfd37d36`::Granted request >Thread-338::DEBUG::2014-05-13 12:41:36,981::task::827::TaskManager.Task::(resourceAcquired) Task=`9b25966c-0081-4a9d-acbd-6cc9f4bf7a54`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:41:36,981::task::990::TaskManager.Task::(_decref) Task=`9b25966c-0081-4a9d-acbd-6cc9f4bf7a54`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:41:36,982::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:41:36,982::task::1185::TaskManager.Task::(prepare) Task=`9b25966c-0081-4a9d-acbd-6cc9f4bf7a54`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:41:36,982::task::595::TaskManager.Task::(_updateState) Task=`9b25966c-0081-4a9d-acbd-6cc9f4bf7a54`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:41:36,982::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:41:36,982::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:41:36,983::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:41:36,983::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:41:36,983::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:41:36,983::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:41:36,983::task::990::TaskManager.Task::(_decref) Task=`9b25966c-0081-4a9d-acbd-6cc9f4bf7a54`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:41:37,093::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:37,169::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307924 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:41:39,151::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:41:39,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:39,263::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0380231 s, 26.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:41:40,275::task::595::TaskManager.Task::(_updateState) Task=`c100691c-64a0-40b7-9f0b-216f311a9bec`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:41:40,275::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:41:40,275::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000734718', 'lastCheck': '7.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:41:40,276::task::1185::TaskManager.Task::(prepare) Task=`c100691c-64a0-40b7-9f0b-216f311a9bec`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000734718', 'lastCheck': '7.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:41:40,276::task::595::TaskManager.Task::(_updateState) Task=`c100691c-64a0-40b7-9f0b-216f311a9bec`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:41:40,276::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:41:40,276::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:41:40,276::task::990::TaskManager.Task::(_decref) Task=`c100691c-64a0-40b7-9f0b-216f311a9bec`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:41:41,276::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:41,355::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364091 s, 28.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:41:42,170::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:41:43,237::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:41:43,244::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000565666 s, 7.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:41:43,368::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:43,739::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.329358 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:41:45,187::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:41:45,752::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:45,830::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.034134 s, 30.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:41:47,001::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1c80df6] >Thread-66::DEBUG::2014-05-13 12:41:47,001::task::595::TaskManager.Task::(_updateState) Task=`5c4b5589-538d-4754-8c7a-6461dab8ee1b`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:41:47,001::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:41:47,002::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:41:47,002::task::1185::TaskManager.Task::(prepare) Task=`5c4b5589-538d-4754-8c7a-6461dab8ee1b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:41:47,002::task::595::TaskManager.Task::(_updateState) Task=`5c4b5589-538d-4754-8c7a-6461dab8ee1b`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:41:47,002::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:41:47,002::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:41:47,003::task::990::TaskManager.Task::(_decref) Task=`5c4b5589-538d-4754-8c7a-6461dab8ee1b`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:41:47,009::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1c80df6] >Thread-338::DEBUG::2014-05-13 12:41:47,009::task::595::TaskManager.Task::(_updateState) Task=`1aecd5b2-04d3-41cc-ae23-8fbcb26dba5d`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:41:47,009::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:41:47,010::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`800cad4f-0a19-44cb-bb05-8767cb5b3ba6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:41:47,010::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:41:47,010::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:41:47,010::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`800cad4f-0a19-44cb-bb05-8767cb5b3ba6`::Granted request >Thread-338::DEBUG::2014-05-13 12:41:47,011::task::827::TaskManager.Task::(resourceAcquired) Task=`1aecd5b2-04d3-41cc-ae23-8fbcb26dba5d`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:41:47,011::task::990::TaskManager.Task::(_decref) Task=`1aecd5b2-04d3-41cc-ae23-8fbcb26dba5d`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:41:47,012::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:41:47,012::task::1185::TaskManager.Task::(prepare) Task=`1aecd5b2-04d3-41cc-ae23-8fbcb26dba5d`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:41:47,012::task::595::TaskManager.Task::(_updateState) Task=`1aecd5b2-04d3-41cc-ae23-8fbcb26dba5d`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:41:47,012::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:41:47,012::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:41:47,013::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:41:47,013::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:41:47,013::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:41:47,013::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:41:47,013::task::990::TaskManager.Task::(_decref) Task=`1aecd5b2-04d3-41cc-ae23-8fbcb26dba5d`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:41:47,843::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:47,923::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374705 s, 27.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:41:48,204::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:41:49,936::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:50,011::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0337326 s, 30.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:41:51,224::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:41:52,024::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:52,098::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307969 s, 33.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:41:53,252::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:41:53,260::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000797027 s, 5.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:41:54,111::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:54,202::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0495504 s, 20.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:41:54,241::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:41:55,410::task::595::TaskManager.Task::(_updateState) Task=`197f75d7-e34a-4113-81fa-acf6a28038a2`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:41:55,410::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:41:55,410::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000797027', 'lastCheck': '2.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:41:55,410::task::1185::TaskManager.Task::(prepare) Task=`197f75d7-e34a-4113-81fa-acf6a28038a2`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000797027', 'lastCheck': '2.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:41:55,410::task::595::TaskManager.Task::(_updateState) Task=`197f75d7-e34a-4113-81fa-acf6a28038a2`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:41:55,410::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:41:55,411::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:41:55,411::task::990::TaskManager.Task::(_decref) Task=`197f75d7-e34a-4113-81fa-acf6a28038a2`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:41:56,215::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:56,295::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381826 s, 26.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:41:57,032::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [12ac301a] >Thread-66::DEBUG::2014-05-13 12:41:57,033::task::595::TaskManager.Task::(_updateState) Task=`d09f632e-539e-4ab5-aded-0e111a51b14e`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:41:57,033::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:41:57,033::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:41:57,033::task::1185::TaskManager.Task::(prepare) Task=`d09f632e-539e-4ab5-aded-0e111a51b14e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:41:57,034::task::595::TaskManager.Task::(_updateState) Task=`d09f632e-539e-4ab5-aded-0e111a51b14e`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:41:57,034::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:41:57,034::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:41:57,034::task::990::TaskManager.Task::(_decref) Task=`d09f632e-539e-4ab5-aded-0e111a51b14e`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:41:57,039::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [12ac301a] >Thread-338::DEBUG::2014-05-13 12:41:57,039::task::595::TaskManager.Task::(_updateState) Task=`4235aa73-d23d-4190-8384-73471785c6ba`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:41:57,039::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:41:57,040::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`271be2d3-ccf1-462c-9b21-d59c09da3fe5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:41:57,040::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:41:57,040::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:41:57,040::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`271be2d3-ccf1-462c-9b21-d59c09da3fe5`::Granted request >Thread-338::DEBUG::2014-05-13 12:41:57,041::task::827::TaskManager.Task::(resourceAcquired) Task=`4235aa73-d23d-4190-8384-73471785c6ba`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:41:57,041::task::990::TaskManager.Task::(_decref) Task=`4235aa73-d23d-4190-8384-73471785c6ba`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:41:57,042::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:41:57,042::task::1185::TaskManager.Task::(prepare) Task=`4235aa73-d23d-4190-8384-73471785c6ba`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:41:57,042::task::595::TaskManager.Task::(_updateState) Task=`4235aa73-d23d-4190-8384-73471785c6ba`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:41:57,042::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:41:57,042::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:41:57,043::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:41:57,043::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:41:57,043::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:41:57,043::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:41:57,043::task::990::TaskManager.Task::(_decref) Task=`4235aa73-d23d-4190-8384-73471785c6ba`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:41:57,259::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:41:58,308::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:41:58,384::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0334882 s, 30.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:42:00,277::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:42:00,397::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:42:00,473::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0340007 s, 30.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:42:02,486::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:42:02,579::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0389323 s, 26.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:42:03,267::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:42:03,275::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000726961 s, 5.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:42:03,297::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:42:04,592::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:42:04,668::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.034041 s, 30.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:42:06,316::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:42:06,681::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:42:06,757::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0337229 s, 30.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:42:07,062::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:42:07,062::task::595::TaskManager.Task::(_updateState) Task=`42469fb7-b082-49be-9720-5e505195e5da`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:42:07,063::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:42:07,063::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:42:07,063::task::1185::TaskManager.Task::(prepare) Task=`42469fb7-b082-49be-9720-5e505195e5da`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:42:07,063::task::595::TaskManager.Task::(_updateState) Task=`42469fb7-b082-49be-9720-5e505195e5da`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:42:07,064::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:42:07,064::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:42:07,064::task::990::TaskManager.Task::(_decref) Task=`42469fb7-b082-49be-9720-5e505195e5da`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:42:07,068::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:42:07,069::task::595::TaskManager.Task::(_updateState) Task=`9c0bc116-e055-40d4-b62d-6d2f3b4298a1`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:42:07,069::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:42:07,069::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3ab18df4-2c38-4697-be08-4647bef48fed`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:42:07,070::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:42:07,070::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:42:07,070::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`3ab18df4-2c38-4697-be08-4647bef48fed`::Granted request >Thread-338::DEBUG::2014-05-13 12:42:07,071::task::827::TaskManager.Task::(resourceAcquired) Task=`9c0bc116-e055-40d4-b62d-6d2f3b4298a1`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:42:07,071::task::990::TaskManager.Task::(_decref) Task=`9c0bc116-e055-40d4-b62d-6d2f3b4298a1`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:42:07,071::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:42:07,072::task::1185::TaskManager.Task::(prepare) Task=`9c0bc116-e055-40d4-b62d-6d2f3b4298a1`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:42:07,072::task::595::TaskManager.Task::(_updateState) Task=`9c0bc116-e055-40d4-b62d-6d2f3b4298a1`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:42:07,072::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:42:07,072::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:42:07,072::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:42:07,073::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:42:07,073::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:42:07,073::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:42:07,073::task::990::TaskManager.Task::(_decref) Task=`9c0bc116-e055-40d4-b62d-6d2f3b4298a1`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:42:08,769::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:42:09,168::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.356317 s, 2.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:42:09,333::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:42:10,556::task::595::TaskManager.Task::(_updateState) Task=`0ae8a411-b8b8-4723-a862-25f54b8c8a89`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:42:10,556::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:42:10,557::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000726961', 'lastCheck': '7.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:42:10,557::task::1185::TaskManager.Task::(prepare) Task=`0ae8a411-b8b8-4723-a862-25f54b8c8a89`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000726961', 'lastCheck': '7.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:42:10,557::task::595::TaskManager.Task::(_updateState) Task=`0ae8a411-b8b8-4723-a862-25f54b8c8a89`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:42:10,557::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:42:10,557::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:42:10,557::task::990::TaskManager.Task::(_decref) Task=`0ae8a411-b8b8-4723-a862-25f54b8c8a89`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:42:11,181::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:42:11,258::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356557 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:42:12,354::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:42:13,272::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 12:42:13,283::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:42:13,299::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000652063 s, 6.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:42:13,382::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0687164 s, 14.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:42:15,372::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:42:15,395::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:42:15,469::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315017 s, 32.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:42:17,091::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:42:17,091::task::595::TaskManager.Task::(_updateState) Task=`1d8bd50c-71b7-4d25-90b7-232faa92a83c`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:42:17,091::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:42:17,092::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:42:17,092::task::1185::TaskManager.Task::(prepare) Task=`1d8bd50c-71b7-4d25-90b7-232faa92a83c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:42:17,092::task::595::TaskManager.Task::(_updateState) Task=`1d8bd50c-71b7-4d25-90b7-232faa92a83c`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:42:17,092::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:42:17,092::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:42:17,093::task::990::TaskManager.Task::(_decref) Task=`1d8bd50c-71b7-4d25-90b7-232faa92a83c`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:42:17,097::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:42:17,098::task::595::TaskManager.Task::(_updateState) Task=`235fb6f8-d326-4449-980c-a602ce69b410`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:42:17,098::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:42:17,098::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5997ba95-d048-4b48-9793-cdd5f66331e1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:42:17,099::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:42:17,099::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:42:17,099::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5997ba95-d048-4b48-9793-cdd5f66331e1`::Granted request >Thread-338::DEBUG::2014-05-13 12:42:17,099::task::827::TaskManager.Task::(resourceAcquired) Task=`235fb6f8-d326-4449-980c-a602ce69b410`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:42:17,100::task::990::TaskManager.Task::(_decref) Task=`235fb6f8-d326-4449-980c-a602ce69b410`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:42:17,100::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:42:17,101::task::1185::TaskManager.Task::(prepare) Task=`235fb6f8-d326-4449-980c-a602ce69b410`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:42:17,101::task::595::TaskManager.Task::(_updateState) Task=`235fb6f8-d326-4449-980c-a602ce69b410`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:42:17,101::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:42:17,101::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:42:17,101::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:42:17,102::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:42:17,102::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:42:17,102::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:42:17,102::task::990::TaskManager.Task::(_decref) Task=`235fb6f8-d326-4449-980c-a602ce69b410`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:42:17,482::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:42:17,561::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358999 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:42:18,389::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:42:19,574::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:42:19,651::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0329215 s, 31.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:42:21,407::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:42:21,664::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:42:22,035::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.327848 s, 3.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:42:23,325::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:42:23,333::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000688326 s, 6.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:42:24,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:42:24,131::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0395248 s, 25.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:42:24,427::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:42:25,689::task::595::TaskManager.Task::(_updateState) Task=`9462c278-e6a4-4fde-9b6d-5c745cfb2bf1`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:42:25,689::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:42:25,690::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000688326', 'lastCheck': '2.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:42:25,690::task::1185::TaskManager.Task::(prepare) Task=`9462c278-e6a4-4fde-9b6d-5c745cfb2bf1`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000688326', 'lastCheck': '2.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:42:25,690::task::595::TaskManager.Task::(_updateState) Task=`9462c278-e6a4-4fde-9b6d-5c745cfb2bf1`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:42:25,690::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:42:25,690::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:42:25,690::task::990::TaskManager.Task::(_decref) Task=`9462c278-e6a4-4fde-9b6d-5c745cfb2bf1`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:42:26,145::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:42:26,225::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387342 s, 26.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:42:27,121::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:42:27,160::task::595::TaskManager.Task::(_updateState) Task=`3bbb51de-1ed4-4326-8c73-6c83df45850a`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:42:27,160::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:42:27,161::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:42:27,161::task::1185::TaskManager.Task::(prepare) Task=`3bbb51de-1ed4-4326-8c73-6c83df45850a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:42:27,161::task::595::TaskManager.Task::(_updateState) Task=`3bbb51de-1ed4-4326-8c73-6c83df45850a`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:42:27,161::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:42:27,161::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:42:27,161::task::990::TaskManager.Task::(_decref) Task=`3bbb51de-1ed4-4326-8c73-6c83df45850a`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:42:27,166::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:42:27,166::task::595::TaskManager.Task::(_updateState) Task=`7378f418-5029-4c22-be5d-061f286c1038`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:42:27,166::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:42:27,167::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`328c0507-0901-4f8e-8952-11ed77162f31`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:42:27,167::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:42:27,167::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:42:27,167::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`328c0507-0901-4f8e-8952-11ed77162f31`::Granted request >Thread-338::DEBUG::2014-05-13 12:42:27,168::task::827::TaskManager.Task::(resourceAcquired) Task=`7378f418-5029-4c22-be5d-061f286c1038`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:42:27,168::task::990::TaskManager.Task::(_decref) Task=`7378f418-5029-4c22-be5d-061f286c1038`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:42:27,169::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:42:27,169::task::1185::TaskManager.Task::(prepare) Task=`7378f418-5029-4c22-be5d-061f286c1038`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:42:27,169::task::595::TaskManager.Task::(_updateState) Task=`7378f418-5029-4c22-be5d-061f286c1038`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:42:27,169::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:42:27,170::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:42:27,170::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:42:27,170::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:42:27,170::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:42:27,170::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:42:27,171::task::990::TaskManager.Task::(_decref) Task=`7378f418-5029-4c22-be5d-061f286c1038`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:42:27,444::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:42:28,238::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:42:28,314::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0337597 s, 30.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:42:30,327::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:42:30,408::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.038664 s, 26.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:42:30,465::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:42:32,417::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:42:32,498::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382653 s, 26.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:42:33,340::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:42:33,348::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000743564 s, 5.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:42:33,487::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:42:34,511::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:42:34,600::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0468937 s, 21.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:42:36,033::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [11fada4e] >Thread-66::DEBUG::2014-05-13 12:42:36,033::task::595::TaskManager.Task::(_updateState) Task=`6b4f4ddd-44a5-4af4-9aaa-44837ff51dfb`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:42:36,034::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.160.108:/RHEV/ogofen/2', 'iqn': '', 'user': '', 'tpgt': '1', 'password': '******', 'id': '00000000-0000-0000-0000-000000000000', 'port': ''}], options=None) >Thread-66::DEBUG::2014-05-13 12:42:36,036::mount::226::Storage.Misc.excCmd::(_runcmd) '/usr/bin/sudo -n /bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6,nfsvers=3 10.35.160.108:/RHEV/ogofen/2 /rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_2' (cwd None) >Thread-66::DEBUG::2014-05-13 12:42:36,123::hsm::2328::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_2 >Thread-66::DEBUG::2014-05-13 12:42:36,125::hsm::2352::Storage.HSM::(__prefetchDomains) Found SD uuids: ('9f4c0902-8dba-44c3-8181-9a6b649c61ff',) >Thread-66::DEBUG::2014-05-13 12:42:36,126::hsm::2408::Storage.HSM::(connectStorageServer) knownSDs: {1ee9ca05-ac9d-4d53-ac4b-6afbe1da0850: storage.nfsSD.findDomain, 1bc6ed6b-8167-4de8-bc1e-ed625a0274d3: storage.blockSD.findDomain, 49db7ad8-2283-4caf-ad27-5dac5b9cb950: storage.blockSD.findDomain, 2874d6f5-ac53-466a-973b-04ddc3beaa24: storage.blockSD.findDomain, 3e8d0026-71e2-43b8-a6cd-853b8861201b: storage.nfsSD.findDomain, 689a6ef7-211a-4bcb-8ac1-42f5cc467967: storage.nfsSD.findDomain, d5af43fc-0c38-4d75-89ad-216daa092503: storage.nfsSD.findDomain, 598916de-8a64-4485-bccb-bc3cbb623569: storage.nfsSD.findDomain, f853f6fe-5b39-4f03-b8a5-dad5823f0bb7: storage.blockSD.findDomain, bfd13673-a430-442d-beb5-dd42d2107b7d: storage.nfsSD.findDomain, 23e7e31a-283a-4398-920a-b3ba8038cb9f: storage.blockSD.findDomain, e4894e28-76eb-4855-bbd1-580c11d4ad51: storage.nfsSD.findDomain, 41830dad-f012-4b4d-b636-863f66d5ca4b: storage.blockSD.findDomain, 6be03a71-472c-4714-bf93-340fdf604955: storage.blockSD.findDomain, c7b37830-fa22-4e63-a8e6-ed70e742a188: storage.nfsSD.findDomain, 9f4c0902-8dba-44c3-8181-9a6b649c61ff: storage.nfsSD.findDomain} >Thread-66::INFO::2014-05-13 12:42:36,126::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-66::DEBUG::2014-05-13 12:42:36,132::task::1185::TaskManager.Task::(prepare) Task=`6b4f4ddd-44a5-4af4-9aaa-44837ff51dfb`::finished: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-66::DEBUG::2014-05-13 12:42:36,132::task::595::TaskManager.Task::(_updateState) Task=`6b4f4ddd-44a5-4af4-9aaa-44837ff51dfb`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:42:36,133::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:42:36,133::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:42:36,133::task::990::TaskManager.Task::(_decref) Task=`6b4f4ddd-44a5-4af4-9aaa-44837ff51dfb`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:42:36,241::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6e71620d] >Thread-66::DEBUG::2014-05-13 12:42:36,242::task::595::TaskManager.Task::(_updateState) Task=`5d4f6403-8b1f-4575-bed9-ebcb36589be7`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:42:36,242::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.160.108:/RHEV/ogofen/2', 'iqn': '', 'user': '', 'tpgt': '1', 'password': '******', 'id': '09bc252b-f212-4653-8c3a-729a76a88f3c', 'port': ''}], options=None) >Thread-66::DEBUG::2014-05-13 12:42:36,245::hsm::2328::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_2 >Thread-66::DEBUG::2014-05-13 12:42:36,246::hsm::2352::Storage.HSM::(__prefetchDomains) Found SD uuids: ('9f4c0902-8dba-44c3-8181-9a6b649c61ff',) >Thread-66::DEBUG::2014-05-13 12:42:36,246::hsm::2408::Storage.HSM::(connectStorageServer) knownSDs: {1ee9ca05-ac9d-4d53-ac4b-6afbe1da0850: storage.nfsSD.findDomain, 1bc6ed6b-8167-4de8-bc1e-ed625a0274d3: storage.blockSD.findDomain, 49db7ad8-2283-4caf-ad27-5dac5b9cb950: storage.blockSD.findDomain, 2874d6f5-ac53-466a-973b-04ddc3beaa24: storage.blockSD.findDomain, 3e8d0026-71e2-43b8-a6cd-853b8861201b: storage.nfsSD.findDomain, 689a6ef7-211a-4bcb-8ac1-42f5cc467967: storage.nfsSD.findDomain, d5af43fc-0c38-4d75-89ad-216daa092503: storage.nfsSD.findDomain, 598916de-8a64-4485-bccb-bc3cbb623569: storage.nfsSD.findDomain, f853f6fe-5b39-4f03-b8a5-dad5823f0bb7: storage.blockSD.findDomain, bfd13673-a430-442d-beb5-dd42d2107b7d: storage.nfsSD.findDomain, 23e7e31a-283a-4398-920a-b3ba8038cb9f: storage.blockSD.findDomain, e4894e28-76eb-4855-bbd1-580c11d4ad51: storage.nfsSD.findDomain, 41830dad-f012-4b4d-b636-863f66d5ca4b: storage.blockSD.findDomain, 6be03a71-472c-4714-bf93-340fdf604955: storage.blockSD.findDomain, c7b37830-fa22-4e63-a8e6-ed70e742a188: storage.nfsSD.findDomain, 9f4c0902-8dba-44c3-8181-9a6b649c61ff: storage.nfsSD.findDomain} >Thread-66::INFO::2014-05-13 12:42:36,246::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '09bc252b-f212-4653-8c3a-729a76a88f3c'}]} >Thread-66::DEBUG::2014-05-13 12:42:36,246::task::1185::TaskManager.Task::(prepare) Task=`5d4f6403-8b1f-4575-bed9-ebcb36589be7`::finished: {'statuslist': [{'status': 0, 'id': '09bc252b-f212-4653-8c3a-729a76a88f3c'}]} >Thread-66::DEBUG::2014-05-13 12:42:36,246::task::595::TaskManager.Task::(_updateState) Task=`5d4f6403-8b1f-4575-bed9-ebcb36589be7`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:42:36,247::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:42:36,247::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:42:36,247::task::990::TaskManager.Task::(_decref) Task=`5d4f6403-8b1f-4575-bed9-ebcb36589be7`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:42:36,256::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6e71620d] >Thread-66::DEBUG::2014-05-13 12:42:36,256::task::595::TaskManager.Task::(_updateState) Task=`2ed561c0-d1c1-421a-8e32-8bb1ddb94d70`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:42:36,256::logUtils::44::dispatcher::(wrapper) Run and protect: createStorageDomain(storageType=1, sdUUID='4d1caccc-b042-4944-abdc-4883bc1f0fad', domainName='NFS_2', typeSpecificArg='10.35.160.108:/RHEV/ogofen/2', domClass=1, domVersion='3', options=None) >Thread-66::DEBUG::2014-05-13 12:42:36,257::misc::762::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-66::DEBUG::2014-05-13 12:42:36,257::misc::764::SamplingMethod::(__call__) Got in to sampling method >Thread-66::DEBUG::2014-05-13 12:42:36,257::misc::762::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-66::DEBUG::2014-05-13 12:42:36,257::misc::764::SamplingMethod::(__call__) Got in to sampling method >Thread-66::DEBUG::2014-05-13 12:42:36,257::iscsi::407::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-66::DEBUG::2014-05-13 12:42:36,257::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-66::DEBUG::2014-05-13 12:42:36,280::misc::772::SamplingMethod::(__call__) Returning last result >Thread-66::DEBUG::2014-05-13 12:42:36,280::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:42:36,514::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:42:36,612::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:42:36,710::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0344816 s, 29.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:42:36,728::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:42:36,729::lvm::497::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-66::DEBUG::2014-05-13 12:42:36,729::lvm::499::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-66::DEBUG::2014-05-13 12:42:36,729::lvm::508::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-66::DEBUG::2014-05-13 12:42:36,730::lvm::510::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-66::DEBUG::2014-05-13 12:42:36,730::lvm::528::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-66::DEBUG::2014-05-13 12:42:36,730::lvm::530::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-66::DEBUG::2014-05-13 12:42:36,730::misc::772::SamplingMethod::(__call__) Returning last result >Thread-66::ERROR::2014-05-13 12:42:36,730::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 4d1caccc-b042-4944-abdc-4883bc1f0fad >Thread-66::ERROR::2014-05-13 12:42:36,731::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 4d1caccc-b042-4944-abdc-4883bc1f0fad >Thread-66::DEBUG::2014-05-13 12:42:36,731::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-66::DEBUG::2014-05-13 12:42:36,733::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 4d1caccc-b042-4944-abdc-4883bc1f0fad' (cwd None) >Thread-66::DEBUG::2014-05-13 12:42:36,958::lvm::296::Storage.Misc.excCmd::(cmd) FAILED: <err> = ' Volume group "4d1caccc-b042-4944-abdc-4883bc1f0fad" not found\n Skipping volume group 4d1caccc-b042-4944-abdc-4883bc1f0fad\n'; <rc> = 5 >Thread-66::WARNING::2014-05-13 12:42:36,960::lvm::378::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "4d1caccc-b042-4944-abdc-4883bc1f0fad" not found', ' Skipping volume group 4d1caccc-b042-4944-abdc-4883bc1f0fad'] >Thread-66::DEBUG::2014-05-13 12:42:36,960::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-66::ERROR::2014-05-13 12:42:37,101::sdc::143::Storage.StorageDomainCache::(_findDomain) domain 4d1caccc-b042-4944-abdc-4883bc1f0fad not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 171, in _findUnfetchedDomain > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('4d1caccc-b042-4944-abdc-4883bc1f0fad',) >Thread-66::INFO::2014-05-13 12:42:37,102::nfsSD::69::Storage.StorageDomain::(create) sdUUID=4d1caccc-b042-4944-abdc-4883bc1f0fad domainName=NFS_2 remotePath=10.35.160.108:/RHEV/ogofen/2 domClass=1 >Thread-13::DEBUG::2014-05-13 12:42:37,191::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2555e502] >Thread-13::DEBUG::2014-05-13 12:42:37,191::task::595::TaskManager.Task::(_updateState) Task=`e9c8928f-61de-4799-82fa-4306d292e6ca`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:42:37,192::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:42:37,192::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:42:37,192::task::1185::TaskManager.Task::(prepare) Task=`e9c8928f-61de-4799-82fa-4306d292e6ca`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:42:37,193::task::595::TaskManager.Task::(_updateState) Task=`e9c8928f-61de-4799-82fa-4306d292e6ca`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:42:37,193::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:42:37,193::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:42:37,193::task::990::TaskManager.Task::(_decref) Task=`e9c8928f-61de-4799-82fa-4306d292e6ca`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:42:37,198::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2555e502] >Thread-338::DEBUG::2014-05-13 12:42:37,198::task::595::TaskManager.Task::(_updateState) Task=`ab330702-5a2f-4847-9c8b-460bb12c909e`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:42:37,199::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:42:37,199::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8b65c917-ff16-4cf7-8230-0247b4e8e43c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:42:37,200::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:42:37,200::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:42:37,200::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8b65c917-ff16-4cf7-8230-0247b4e8e43c`::Granted request >Thread-338::DEBUG::2014-05-13 12:42:37,201::task::827::TaskManager.Task::(resourceAcquired) Task=`ab330702-5a2f-4847-9c8b-460bb12c909e`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:42:37,201::task::990::TaskManager.Task::(_decref) Task=`ab330702-5a2f-4847-9c8b-460bb12c909e`::ref 1 aborting False >Thread-338::DEBUG::2014-05-13 12:42:37,202::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:42:37,203::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-66::ERROR::2014-05-13 12:42:37,266::task::866::TaskManager.Task::(_setError) Task=`2ed561c0-d1c1-421a-8e32-8bb1ddb94d70`::Unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/storage/task.py", line 873, in _run > return fn(*args, **kargs) > File "/usr/share/vdsm/logUtils.py", line 45, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/storage/hsm.py", line 2631, in createStorageDomain > domVersion) > File "/usr/share/vdsm/storage/nfsSD.py", line 80, in create > version) > File "/usr/share/vdsm/storage/nfsSD.py", line 54, in _preCreateValidation > raise se.StorageDomainNotEmpty(typeSpecificArg) >StorageDomainNotEmpty: Storage domain is not empty - requires cleaning: ('10.35.160.108:/RHEV/ogofen/2',) >Thread-66::DEBUG::2014-05-13 12:42:37,267::task::885::TaskManager.Task::(_run) Task=`2ed561c0-d1c1-421a-8e32-8bb1ddb94d70`::Task._run: 2ed561c0-d1c1-421a-8e32-8bb1ddb94d70 (1, '4d1caccc-b042-4944-abdc-4883bc1f0fad', 'NFS_2', '10.35.160.108:/RHEV/ogofen/2', 1, '3') {} failed - stopping task >Thread-66::DEBUG::2014-05-13 12:42:37,267::task::1211::TaskManager.Task::(stop) Task=`2ed561c0-d1c1-421a-8e32-8bb1ddb94d70`::stopping in state preparing (force False) >Thread-66::DEBUG::2014-05-13 12:42:37,267::task::990::TaskManager.Task::(_decref) Task=`2ed561c0-d1c1-421a-8e32-8bb1ddb94d70`::ref 1 aborting True >Thread-66::INFO::2014-05-13 12:42:37,268::task::1168::TaskManager.Task::(prepare) Task=`2ed561c0-d1c1-421a-8e32-8bb1ddb94d70`::aborting: Task is aborted: 'Storage domain is not empty - requires cleaning' - code 361 >Thread-66::DEBUG::2014-05-13 12:42:37,268::task::1173::TaskManager.Task::(prepare) Task=`2ed561c0-d1c1-421a-8e32-8bb1ddb94d70`::Prepare: aborted: Storage domain is not empty - requires cleaning >Thread-66::DEBUG::2014-05-13 12:42:37,268::task::990::TaskManager.Task::(_decref) Task=`2ed561c0-d1c1-421a-8e32-8bb1ddb94d70`::ref 0 aborting True >Thread-66::DEBUG::2014-05-13 12:42:37,268::task::925::TaskManager.Task::(_doAbort) Task=`2ed561c0-d1c1-421a-8e32-8bb1ddb94d70`::Task._doAbort: force False >Thread-66::DEBUG::2014-05-13 12:42:37,268::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:42:37,269::task::595::TaskManager.Task::(_updateState) Task=`2ed561c0-d1c1-421a-8e32-8bb1ddb94d70`::moving from state preparing -> state aborting >Thread-66::DEBUG::2014-05-13 12:42:37,269::task::550::TaskManager.Task::(__state_aborting) Task=`2ed561c0-d1c1-421a-8e32-8bb1ddb94d70`::_aborting: recover policy none >Thread-66::DEBUG::2014-05-13 12:42:37,269::task::595::TaskManager.Task::(_updateState) Task=`2ed561c0-d1c1-421a-8e32-8bb1ddb94d70`::moving from state aborting -> state failed >Thread-66::DEBUG::2014-05-13 12:42:37,269::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:42:37,269::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::ERROR::2014-05-13 12:42:37,270::dispatcher::65::Storage.Dispatcher.Protect::(run) {'status': {'message': "Storage domain is not empty - requires cleaning: ('10.35.160.108:/RHEV/ogofen/2',)", 'code': 361}} >Thread-338::DEBUG::2014-05-13 12:42:37,383::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:42:37,384::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-338::INFO::2014-05-13 12:42:37,385::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:42:37,385::task::1185::TaskManager.Task::(prepare) Task=`ab330702-5a2f-4847-9c8b-460bb12c909e`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:42:37,385::task::595::TaskManager.Task::(_updateState) Task=`ab330702-5a2f-4847-9c8b-460bb12c909e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:42:37,386::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:42:37,386::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:42:37,386::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:42:37,386::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:42:37,387::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:42:37,387::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:42:37,387::task::990::TaskManager.Task::(_decref) Task=`ab330702-5a2f-4847-9c8b-460bb12c909e`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:42:37,513::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [7aa1187d] >Thread-66::DEBUG::2014-05-13 12:42:37,578::task::595::TaskManager.Task::(_updateState) Task=`418dba93-aa53-4a10-8dec-a6203e1617b8`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:42:37,578::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.160.108:/RHEV/ogofen/2', 'iqn': '', 'user': '', 'tpgt': '1', 'password': '******', 'id': '09bc252b-f212-4653-8c3a-729a76a88f3c', 'port': ''}], options=None) >Thread-66::DEBUG::2014-05-13 12:42:37,579::mount::226::Storage.Misc.excCmd::(_runcmd) '/usr/bin/sudo -n /bin/umount -f -l /rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_2' (cwd None) >Thread-66::DEBUG::2014-05-13 12:42:37,631::misc::762::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-66::DEBUG::2014-05-13 12:42:37,632::misc::764::SamplingMethod::(__call__) Got in to sampling method >Thread-66::DEBUG::2014-05-13 12:42:37,632::misc::762::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-66::DEBUG::2014-05-13 12:42:37,632::misc::764::SamplingMethod::(__call__) Got in to sampling method >Thread-66::DEBUG::2014-05-13 12:42:37,632::iscsi::407::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-66::DEBUG::2014-05-13 12:42:37,633::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-66::DEBUG::2014-05-13 12:42:37,655::misc::772::SamplingMethod::(__call__) Returning last result >Thread-66::DEBUG::2014-05-13 12:42:37,655::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >Thread-66::DEBUG::2014-05-13 12:42:37,859::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:42:37,860::lvm::497::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-66::DEBUG::2014-05-13 12:42:37,861::lvm::499::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-66::DEBUG::2014-05-13 12:42:37,861::lvm::508::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-66::DEBUG::2014-05-13 12:42:37,862::lvm::510::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-66::DEBUG::2014-05-13 12:42:37,862::lvm::528::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-66::DEBUG::2014-05-13 12:42:37,863::lvm::530::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-66::DEBUG::2014-05-13 12:42:37,863::misc::772::SamplingMethod::(__call__) Returning last result >Thread-66::INFO::2014-05-13 12:42:37,864::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '09bc252b-f212-4653-8c3a-729a76a88f3c'}]} >Thread-66::DEBUG::2014-05-13 12:42:37,864::task::1185::TaskManager.Task::(prepare) Task=`418dba93-aa53-4a10-8dec-a6203e1617b8`::finished: {'statuslist': [{'status': 0, 'id': '09bc252b-f212-4653-8c3a-729a76a88f3c'}]} >Thread-66::DEBUG::2014-05-13 12:42:37,865::task::595::TaskManager.Task::(_updateState) Task=`418dba93-aa53-4a10-8dec-a6203e1617b8`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:42:37,865::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:42:37,866::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:42:37,866::task::990::TaskManager.Task::(_decref) Task=`418dba93-aa53-4a10-8dec-a6203e1617b8`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:42:38,724::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:42:38,808::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0392293 s, 26.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:42:39,533::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:42:40,820::task::595::TaskManager.Task::(_updateState) Task=`d3da5e45-e661-4843-a2e0-9795c15e6236`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:42:40,820::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:42:40,820::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000743564', 'lastCheck': '7.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:42:40,820::task::1185::TaskManager.Task::(prepare) Task=`d3da5e45-e661-4843-a2e0-9795c15e6236`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000743564', 'lastCheck': '7.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:42:40,820::task::595::TaskManager.Task::(_updateState) Task=`d3da5e45-e661-4843-a2e0-9795c15e6236`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:42:40,821::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:42:40,821::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Dummy-337::DEBUG::2014-05-13 12:42:40,821::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 12:42:40,821::task::990::TaskManager.Task::(_decref) Task=`d3da5e45-e661-4843-a2e0-9795c15e6236`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:42:40,899::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0333005 s, 30.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:42:42,557::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:42:42,912::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 12:42:42,979::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [151cfb7c] >Thread-66::DEBUG::2014-05-13 12:42:42,981::task::595::TaskManager.Task::(_updateState) Task=`cd5c50fa-8a43-4edd-9aef-f8bda8dd6d8b`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:42:42,982::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.160.108:/RHEV/ogofen/3', 'iqn': '', 'user': '', 'tpgt': '1', 'password': '******', 'id': '00000000-0000-0000-0000-000000000000', 'port': ''}], options=None) >Thread-66::DEBUG::2014-05-13 12:42:42,996::mount::226::Storage.Misc.excCmd::(_runcmd) '/usr/bin/sudo -n /bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6,nfsvers=3 10.35.160.108:/RHEV/ogofen/3 /rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:42:43,021::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0324445 s, 31.6 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:42:43,058::hsm::2328::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3 >Thread-66::DEBUG::2014-05-13 12:42:43,060::hsm::2352::Storage.HSM::(__prefetchDomains) Found SD uuids: () >Thread-66::DEBUG::2014-05-13 12:42:43,060::hsm::2408::Storage.HSM::(connectStorageServer) knownSDs: {1ee9ca05-ac9d-4d53-ac4b-6afbe1da0850: storage.nfsSD.findDomain, 1bc6ed6b-8167-4de8-bc1e-ed625a0274d3: storage.blockSD.findDomain, 49db7ad8-2283-4caf-ad27-5dac5b9cb950: storage.blockSD.findDomain, 2874d6f5-ac53-466a-973b-04ddc3beaa24: storage.blockSD.findDomain, 3e8d0026-71e2-43b8-a6cd-853b8861201b: storage.nfsSD.findDomain, 689a6ef7-211a-4bcb-8ac1-42f5cc467967: storage.nfsSD.findDomain, d5af43fc-0c38-4d75-89ad-216daa092503: storage.nfsSD.findDomain, 598916de-8a64-4485-bccb-bc3cbb623569: storage.nfsSD.findDomain, f853f6fe-5b39-4f03-b8a5-dad5823f0bb7: storage.blockSD.findDomain, bfd13673-a430-442d-beb5-dd42d2107b7d: storage.nfsSD.findDomain, 23e7e31a-283a-4398-920a-b3ba8038cb9f: storage.blockSD.findDomain, e4894e28-76eb-4855-bbd1-580c11d4ad51: storage.nfsSD.findDomain, 41830dad-f012-4b4d-b636-863f66d5ca4b: storage.blockSD.findDomain, 6be03a71-472c-4714-bf93-340fdf604955: storage.blockSD.findDomain, c7b37830-fa22-4e63-a8e6-ed70e742a188: storage.nfsSD.findDomain, 9f4c0902-8dba-44c3-8181-9a6b649c61ff: storage.nfsSD.findDomain} >Thread-66::INFO::2014-05-13 12:42:43,060::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-66::DEBUG::2014-05-13 12:42:43,061::task::1185::TaskManager.Task::(prepare) Task=`cd5c50fa-8a43-4edd-9aef-f8bda8dd6d8b`::finished: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-66::DEBUG::2014-05-13 12:42:43,061::task::595::TaskManager.Task::(_updateState) Task=`cd5c50fa-8a43-4edd-9aef-f8bda8dd6d8b`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:42:43,061::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:42:43,061::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:42:43,061::task::990::TaskManager.Task::(_decref) Task=`cd5c50fa-8a43-4edd-9aef-f8bda8dd6d8b`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:42:43,155::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3b4a76ef] >Thread-66::DEBUG::2014-05-13 12:42:43,156::task::595::TaskManager.Task::(_updateState) Task=`5559f1dd-a779-4c5f-915b-1630ac9f3d91`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:42:43,156::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.160.108:/RHEV/ogofen/3', 'iqn': '', 'user': '', 'tpgt': '1', 'password': '******', 'id': '3a6bb180-87f9-441b-a52a-530eb2e61642', 'port': ''}], options=None) >Thread-66::DEBUG::2014-05-13 12:42:43,158::hsm::2328::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3 >Thread-66::DEBUG::2014-05-13 12:42:43,159::hsm::2352::Storage.HSM::(__prefetchDomains) Found SD uuids: () >Thread-66::DEBUG::2014-05-13 12:42:43,159::hsm::2408::Storage.HSM::(connectStorageServer) knownSDs: {1ee9ca05-ac9d-4d53-ac4b-6afbe1da0850: storage.nfsSD.findDomain, 1bc6ed6b-8167-4de8-bc1e-ed625a0274d3: storage.blockSD.findDomain, 49db7ad8-2283-4caf-ad27-5dac5b9cb950: storage.blockSD.findDomain, 2874d6f5-ac53-466a-973b-04ddc3beaa24: storage.blockSD.findDomain, 3e8d0026-71e2-43b8-a6cd-853b8861201b: storage.nfsSD.findDomain, 689a6ef7-211a-4bcb-8ac1-42f5cc467967: storage.nfsSD.findDomain, d5af43fc-0c38-4d75-89ad-216daa092503: storage.nfsSD.findDomain, 598916de-8a64-4485-bccb-bc3cbb623569: storage.nfsSD.findDomain, f853f6fe-5b39-4f03-b8a5-dad5823f0bb7: storage.blockSD.findDomain, bfd13673-a430-442d-beb5-dd42d2107b7d: storage.nfsSD.findDomain, 23e7e31a-283a-4398-920a-b3ba8038cb9f: storage.blockSD.findDomain, e4894e28-76eb-4855-bbd1-580c11d4ad51: storage.nfsSD.findDomain, 41830dad-f012-4b4d-b636-863f66d5ca4b: storage.blockSD.findDomain, 6be03a71-472c-4714-bf93-340fdf604955: storage.blockSD.findDomain, c7b37830-fa22-4e63-a8e6-ed70e742a188: storage.nfsSD.findDomain, 9f4c0902-8dba-44c3-8181-9a6b649c61ff: storage.nfsSD.findDomain} >Thread-66::INFO::2014-05-13 12:42:43,160::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '3a6bb180-87f9-441b-a52a-530eb2e61642'}]} >Thread-66::DEBUG::2014-05-13 12:42:43,160::task::1185::TaskManager.Task::(prepare) Task=`5559f1dd-a779-4c5f-915b-1630ac9f3d91`::finished: {'statuslist': [{'status': 0, 'id': '3a6bb180-87f9-441b-a52a-530eb2e61642'}]} >Thread-66::DEBUG::2014-05-13 12:42:43,160::task::595::TaskManager.Task::(_updateState) Task=`5559f1dd-a779-4c5f-915b-1630ac9f3d91`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:42:43,160::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:42:43,160::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:42:43,160::task::990::TaskManager.Task::(_decref) Task=`5559f1dd-a779-4c5f-915b-1630ac9f3d91`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:42:43,167::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3b4a76ef] >Thread-66::DEBUG::2014-05-13 12:42:43,168::task::595::TaskManager.Task::(_updateState) Task=`054a31ad-0f5e-432f-bdeb-97148f2fca86`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:42:43,168::logUtils::44::dispatcher::(wrapper) Run and protect: createStorageDomain(storageType=1, sdUUID='572b0496-5eab-41cd-a976-4514f111898e', domainName='NFS_2', typeSpecificArg='10.35.160.108:/RHEV/ogofen/3', domClass=1, domVersion='3', options=None) >Thread-66::DEBUG::2014-05-13 12:42:43,168::misc::762::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-66::DEBUG::2014-05-13 12:42:43,168::misc::764::SamplingMethod::(__call__) Got in to sampling method >Thread-66::DEBUG::2014-05-13 12:42:43,169::misc::762::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-66::DEBUG::2014-05-13 12:42:43,169::misc::764::SamplingMethod::(__call__) Got in to sampling method >Thread-66::DEBUG::2014-05-13 12:42:43,169::iscsi::407::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-66::DEBUG::2014-05-13 12:42:43,169::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-66::DEBUG::2014-05-13 12:42:43,192::misc::772::SamplingMethod::(__call__) Returning last result >Thread-66::DEBUG::2014-05-13 12:42:43,193::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >Thread-322::DEBUG::2014-05-13 12:42:43,353::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:42:43,359::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-66::DEBUG::2014-05-13 12:42:43,447::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:42:43,449::lvm::497::OperationMutex::(_invalidateAllPvs) Operation 'lvm reload operation' is holding the operation mutex, waiting... >Thread-322::DEBUG::2014-05-13 12:42:43,619::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:42:43,621::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-66::DEBUG::2014-05-13 12:42:43,622::lvm::497::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-66::DEBUG::2014-05-13 12:42:43,623::lvm::499::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-66::DEBUG::2014-05-13 12:42:43,625::lvm::508::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-66::DEBUG::2014-05-13 12:42:43,627::lvm::510::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-66::DEBUG::2014-05-13 12:42:43,628::lvm::528::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-66::DEBUG::2014-05-13 12:42:43,630::lvm::530::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-66::DEBUG::2014-05-13 12:42:43,632::misc::772::SamplingMethod::(__call__) Returning last result >Thread-66::ERROR::2014-05-13 12:42:43,633::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 572b0496-5eab-41cd-a976-4514f111898e >Thread-66::ERROR::2014-05-13 12:42:43,633::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 572b0496-5eab-41cd-a976-4514f111898e >Thread-66::DEBUG::2014-05-13 12:42:43,635::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:42:43,638::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:42:43,652::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000639309 s, 6.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:42:43,654::lvm::373::OperationMutex::(_reloadvgs) Got the operational mutex >Thread-66::DEBUG::2014-05-13 12:42:43,654::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 572b0496-5eab-41cd-a976-4514f111898e' (cwd None) >Thread-322::DEBUG::2014-05-13 12:42:43,663::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 12:42:43,848::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:42:43,934::lvm::296::Storage.Misc.excCmd::(cmd) FAILED: <err> = ' Volume group "572b0496-5eab-41cd-a976-4514f111898e" not found\n Skipping volume group 572b0496-5eab-41cd-a976-4514f111898e\n'; <rc> = 5 >Thread-66::WARNING::2014-05-13 12:42:43,936::lvm::378::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "572b0496-5eab-41cd-a976-4514f111898e" not found', ' Skipping volume group 572b0496-5eab-41cd-a976-4514f111898e'] >Thread-66::DEBUG::2014-05-13 12:42:43,936::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-66::ERROR::2014-05-13 12:42:44,076::sdc::143::Storage.StorageDomainCache::(_findDomain) domain 572b0496-5eab-41cd-a976-4514f111898e not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 171, in _findUnfetchedDomain > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('572b0496-5eab-41cd-a976-4514f111898e',) >Thread-66::INFO::2014-05-13 12:42:44,076::nfsSD::69::Storage.StorageDomain::(create) sdUUID=572b0496-5eab-41cd-a976-4514f111898e domainName=NFS_2 remotePath=10.35.160.108:/RHEV/ogofen/3 domClass=1 >Thread-66::DEBUG::2014-05-13 12:42:44,238::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-66::DEBUG::2014-05-13 12:42:44,243::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=[] >Thread-66::WARNING::2014-05-13 12:42:44,243::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-66::DEBUG::2014-05-13 12:42:44,244::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-66::DEBUG::2014-05-13 12:42:44,244::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >Thread-66::DEBUG::2014-05-13 12:42:44,244::persistentDict::299::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=NFS_2', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.160.108:/RHEV/ogofen/3', 'ROLE=Regular', 'SDUUID=572b0496-5eab-41cd-a976-4514f111898e', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=06d8db54b521c956db503c23b9077be197d58617'] >Thread-66::DEBUG::2014-05-13 12:42:44,247::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-66::DEBUG::2014-05-13 12:42:44,248::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e >Thread-66::DEBUG::2014-05-13 12:42:44,250::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-66::DEBUG::2014-05-13 12:42:44,257::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=NFS_2', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.160.108:/RHEV/ogofen/3', 'ROLE=Regular', 'SDUUID=572b0496-5eab-41cd-a976-4514f111898e', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=06d8db54b521c956db503c23b9077be197d58617'] >Thread-66::DEBUG::2014-05-13 12:42:44,259::fileSD::609::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-66::DEBUG::2014-05-13 12:42:44,259::resourceManager::421::ResourceManager::(registerNamespace) Registering namespace '572b0496-5eab-41cd-a976-4514f111898e_imageNS' >Thread-66::DEBUG::2014-05-13 12:42:44,260::resourceManager::421::ResourceManager::(registerNamespace) Registering namespace '572b0496-5eab-41cd-a976-4514f111898e_volumeNS' >Thread-66::DEBUG::2014-05-13 12:42:44,260::clusterlock::144::initSANLock::(initSANLock) Initializing SANLock for domain 572b0496-5eab-41cd-a976-4514f111898e >Thread-66::DEBUG::2014-05-13 12:42:44,289::sd::434::Storage.StorageDomain::(initSPMlease) lease initialized successfully >Thread-66::DEBUG::2014-05-13 12:42:44,289::hsm::2647::Storage.HSM::(createStorageDomain) knownSDs: {1ee9ca05-ac9d-4d53-ac4b-6afbe1da0850: storage.nfsSD.findDomain, 1bc6ed6b-8167-4de8-bc1e-ed625a0274d3: storage.blockSD.findDomain, 49db7ad8-2283-4caf-ad27-5dac5b9cb950: storage.blockSD.findDomain, 2874d6f5-ac53-466a-973b-04ddc3beaa24: storage.blockSD.findDomain, 3e8d0026-71e2-43b8-a6cd-853b8861201b: storage.nfsSD.findDomain, 689a6ef7-211a-4bcb-8ac1-42f5cc467967: storage.nfsSD.findDomain, d5af43fc-0c38-4d75-89ad-216daa092503: storage.nfsSD.findDomain, 598916de-8a64-4485-bccb-bc3cbb623569: storage.nfsSD.findDomain, f853f6fe-5b39-4f03-b8a5-dad5823f0bb7: storage.blockSD.findDomain, bfd13673-a430-442d-beb5-dd42d2107b7d: storage.nfsSD.findDomain, 572b0496-5eab-41cd-a976-4514f111898e: storage.nfsSD.findDomain, 23e7e31a-283a-4398-920a-b3ba8038cb9f: storage.blockSD.findDomain, e4894e28-76eb-4855-bbd1-580c11d4ad51: storage.nfsSD.findDomain, 41830dad-f012-4b4d-b636-863f66d5ca4b: storage.blockSD.findDomain, 6be03a71-472c-4714-bf93-340fdf604955: storage.blockSD.findDomain, c7b37830-fa22-4e63-a8e6-ed70e742a188: storage.nfsSD.findDomain, 9f4c0902-8dba-44c3-8181-9a6b649c61ff: storage.nfsSD.findDomain} >Thread-66::INFO::2014-05-13 12:42:44,289::logUtils::47::dispatcher::(wrapper) Run and protect: createStorageDomain, Return response: None >Thread-66::DEBUG::2014-05-13 12:42:44,289::task::1185::TaskManager.Task::(prepare) Task=`054a31ad-0f5e-432f-bdeb-97148f2fca86`::finished: None >Thread-66::DEBUG::2014-05-13 12:42:44,289::task::595::TaskManager.Task::(_updateState) Task=`054a31ad-0f5e-432f-bdeb-97148f2fca86`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:42:44,290::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:42:44,290::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:42:44,290::task::990::TaskManager.Task::(_decref) Task=`054a31ad-0f5e-432f-bdeb-97148f2fca86`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:42:44,336::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3b4a76ef] >Thread-66::DEBUG::2014-05-13 12:42:44,336::task::595::TaskManager.Task::(_updateState) Task=`70a13cb4-a343-41f4-9581-eb246741be97`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:42:44,337::logUtils::44::dispatcher::(wrapper) Run and protect: getStorageDomainStats(sdUUID='572b0496-5eab-41cd-a976-4514f111898e', options=None) >Thread-66::DEBUG::2014-05-13 12:42:44,337::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.572b0496-5eab-41cd-a976-4514f111898e`ReqID=`fc1e8f80-6025-4d7a-85f9-2572fbf3cb8a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2786' at 'getStorageDomainStats' >Thread-66::DEBUG::2014-05-13 12:42:44,337::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' for lock type 'shared' >Thread-66::DEBUG::2014-05-13 12:42:44,338::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' is free. Now locking as 'shared' (1 active user) >Thread-66::DEBUG::2014-05-13 12:42:44,338::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.572b0496-5eab-41cd-a976-4514f111898e`ReqID=`fc1e8f80-6025-4d7a-85f9-2572fbf3cb8a`::Granted request >Thread-66::DEBUG::2014-05-13 12:42:44,339::task::827::TaskManager.Task::(resourceAcquired) Task=`70a13cb4-a343-41f4-9581-eb246741be97`::_resourcesAcquired: Storage.572b0496-5eab-41cd-a976-4514f111898e (shared) >Thread-66::DEBUG::2014-05-13 12:42:44,339::task::990::TaskManager.Task::(_decref) Task=`70a13cb4-a343-41f4-9581-eb246741be97`::ref 1 aborting False >Thread-66::INFO::2014-05-13 12:42:44,340::logUtils::47::dispatcher::(wrapper) Run and protect: getStorageDomainStats, Return response: {'stats': {'mdasize': 0, 'mdathreshold': True, 'mdavalid': True, 'diskfree': '1923395420160', 'disktotal': '2163246071808', 'mdafree': 0}} >Thread-66::DEBUG::2014-05-13 12:42:44,341::task::1185::TaskManager.Task::(prepare) Task=`70a13cb4-a343-41f4-9581-eb246741be97`::finished: {'stats': {'mdasize': 0, 'mdathreshold': True, 'mdavalid': True, 'diskfree': '1923395420160', 'disktotal': '2163246071808', 'mdafree': 0}} >Thread-66::DEBUG::2014-05-13 12:42:44,341::task::595::TaskManager.Task::(_updateState) Task=`70a13cb4-a343-41f4-9581-eb246741be97`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:42:44,341::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.572b0496-5eab-41cd-a976-4514f111898e': < ResourceRef 'Storage.572b0496-5eab-41cd-a976-4514f111898e', isValid: 'True' obj: 'None'>} >Thread-66::DEBUG::2014-05-13 12:42:44,341::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:42:44,341::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' >Thread-66::DEBUG::2014-05-13 12:42:44,342::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' (0 active users) >Thread-66::DEBUG::2014-05-13 12:42:44,342::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' is free, finding out if anyone is waiting for it. >Thread-66::DEBUG::2014-05-13 12:42:44,342::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e', Clearing records. >Thread-66::DEBUG::2014-05-13 12:42:44,342::task::990::TaskManager.Task::(_decref) Task=`70a13cb4-a343-41f4-9581-eb246741be97`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:42:44,495::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [551b7198] >Thread-66::DEBUG::2014-05-13 12:42:44,495::task::595::TaskManager.Task::(_updateState) Task=`44576a98-2e84-4f5b-8964-6a4a40026a72`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:42:44,495::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.160.108:/RHEV/ogofen/3', 'iqn': '', 'user': '', 'tpgt': '1', 'password': '******', 'id': '3a6bb180-87f9-441b-a52a-530eb2e61642', 'port': ''}], options=None) >Thread-66::DEBUG::2014-05-13 12:42:44,496::mount::226::Storage.Misc.excCmd::(_runcmd) '/usr/bin/sudo -n /bin/umount -f -l /rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3' (cwd None) >Thread-66::DEBUG::2014-05-13 12:42:44,538::misc::762::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-66::DEBUG::2014-05-13 12:42:44,538::misc::764::SamplingMethod::(__call__) Got in to sampling method >Thread-66::DEBUG::2014-05-13 12:42:44,538::misc::762::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-66::DEBUG::2014-05-13 12:42:44,538::misc::764::SamplingMethod::(__call__) Got in to sampling method >Thread-66::DEBUG::2014-05-13 12:42:44,539::iscsi::407::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-66::DEBUG::2014-05-13 12:42:44,539::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-66::DEBUG::2014-05-13 12:42:44,561::misc::772::SamplingMethod::(__call__) Returning last result >Thread-66::DEBUG::2014-05-13 12:42:44,562::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >Thread-66::DEBUG::2014-05-13 12:42:44,760::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:42:44,761::lvm::497::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-66::DEBUG::2014-05-13 12:42:44,761::lvm::499::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-66::DEBUG::2014-05-13 12:42:44,762::lvm::508::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-66::DEBUG::2014-05-13 12:42:44,762::lvm::510::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-66::DEBUG::2014-05-13 12:42:44,763::lvm::528::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-66::DEBUG::2014-05-13 12:42:44,763::lvm::530::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-66::DEBUG::2014-05-13 12:42:44,764::misc::772::SamplingMethod::(__call__) Returning last result >Thread-66::INFO::2014-05-13 12:42:44,764::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '3a6bb180-87f9-441b-a52a-530eb2e61642'}]} >Thread-66::DEBUG::2014-05-13 12:42:44,765::task::1185::TaskManager.Task::(prepare) Task=`44576a98-2e84-4f5b-8964-6a4a40026a72`::finished: {'statuslist': [{'status': 0, 'id': '3a6bb180-87f9-441b-a52a-530eb2e61642'}]} >Thread-66::DEBUG::2014-05-13 12:42:44,765::task::595::TaskManager.Task::(_updateState) Task=`44576a98-2e84-4f5b-8964-6a4a40026a72`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:42:44,765::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:42:44,766::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:42:44,766::task::990::TaskManager.Task::(_decref) Task=`44576a98-2e84-4f5b-8964-6a4a40026a72`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:42:44,937::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4010eac9] >Thread-66::DEBUG::2014-05-13 12:42:44,938::task::595::TaskManager.Task::(_updateState) Task=`a8589acf-37af-4bec-8d3f-770bcd9d2aa4`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:42:44,938::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.160.108:/RHEV/ogofen/3', 'iqn': '', 'user': '', 'tpgt': '1', 'password': '******', 'id': '3a6bb180-87f9-441b-a52a-530eb2e61642', 'port': ''}], options=None) >Thread-66::DEBUG::2014-05-13 12:42:44,941::mount::226::Storage.Misc.excCmd::(_runcmd) '/usr/bin/sudo -n /bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6,nfsvers=3 10.35.160.108:/RHEV/ogofen/3 /rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3' (cwd None) >Thread-66::DEBUG::2014-05-13 12:42:44,977::hsm::2328::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3 >Thread-66::DEBUG::2014-05-13 12:42:44,979::hsm::2352::Storage.HSM::(__prefetchDomains) Found SD uuids: ('572b0496-5eab-41cd-a976-4514f111898e',) >Thread-66::DEBUG::2014-05-13 12:42:44,979::hsm::2408::Storage.HSM::(connectStorageServer) knownSDs: {1ee9ca05-ac9d-4d53-ac4b-6afbe1da0850: storage.nfsSD.findDomain, 1bc6ed6b-8167-4de8-bc1e-ed625a0274d3: storage.blockSD.findDomain, 49db7ad8-2283-4caf-ad27-5dac5b9cb950: storage.blockSD.findDomain, 2874d6f5-ac53-466a-973b-04ddc3beaa24: storage.blockSD.findDomain, 3e8d0026-71e2-43b8-a6cd-853b8861201b: storage.nfsSD.findDomain, 689a6ef7-211a-4bcb-8ac1-42f5cc467967: storage.nfsSD.findDomain, d5af43fc-0c38-4d75-89ad-216daa092503: storage.nfsSD.findDomain, 598916de-8a64-4485-bccb-bc3cbb623569: storage.nfsSD.findDomain, f853f6fe-5b39-4f03-b8a5-dad5823f0bb7: storage.blockSD.findDomain, bfd13673-a430-442d-beb5-dd42d2107b7d: storage.nfsSD.findDomain, 572b0496-5eab-41cd-a976-4514f111898e: storage.nfsSD.findDomain, 23e7e31a-283a-4398-920a-b3ba8038cb9f: storage.blockSD.findDomain, e4894e28-76eb-4855-bbd1-580c11d4ad51: storage.nfsSD.findDomain, 41830dad-f012-4b4d-b636-863f66d5ca4b: storage.blockSD.findDomain, 6be03a71-472c-4714-bf93-340fdf604955: storage.blockSD.findDomain, c7b37830-fa22-4e63-a8e6-ed70e742a188: storage.nfsSD.findDomain, 9f4c0902-8dba-44c3-8181-9a6b649c61ff: storage.nfsSD.findDomain} >Thread-66::INFO::2014-05-13 12:42:44,980::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '3a6bb180-87f9-441b-a52a-530eb2e61642'}]} >Thread-66::DEBUG::2014-05-13 12:42:44,980::task::1185::TaskManager.Task::(prepare) Task=`a8589acf-37af-4bec-8d3f-770bcd9d2aa4`::finished: {'statuslist': [{'status': 0, 'id': '3a6bb180-87f9-441b-a52a-530eb2e61642'}]} >Thread-66::DEBUG::2014-05-13 12:42:44,980::task::595::TaskManager.Task::(_updateState) Task=`a8589acf-37af-4bec-8d3f-770bcd9d2aa4`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:42:44,980::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:42:44,981::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:42:44,981::task::990::TaskManager.Task::(_decref) Task=`a8589acf-37af-4bec-8d3f-770bcd9d2aa4`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:42:44,994::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [156b68c1] >Thread-338::DEBUG::2014-05-13 12:42:44,995::task::595::TaskManager.Task::(_updateState) Task=`1a868a14-4e2d-4257-b6f2-7c938dd7211f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:42:44,995::logUtils::44::dispatcher::(wrapper) Run and protect: attachStorageDomain(sdUUID='572b0496-5eab-41cd-a976-4514f111898e', spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:42:44,995::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e79ce080-ea2c-4bd1-ac03-d602090fdc7e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1197' at 'attachStorageDomain' >Thread-338::DEBUG::2014-05-13 12:42:44,996::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'exclusive' >Thread-338::DEBUG::2014-05-13 12:42:44,996::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'exclusive' (1 active user) >Thread-338::DEBUG::2014-05-13 12:42:44,996::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e79ce080-ea2c-4bd1-ac03-d602090fdc7e`::Granted request >Thread-338::DEBUG::2014-05-13 12:42:44,996::task::827::TaskManager.Task::(resourceAcquired) Task=`1a868a14-4e2d-4257-b6f2-7c938dd7211f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (exclusive) >Thread-338::DEBUG::2014-05-13 12:42:44,997::task::990::TaskManager.Task::(_decref) Task=`1a868a14-4e2d-4257-b6f2-7c938dd7211f`::ref 1 aborting False >Thread-338::DEBUG::2014-05-13 12:42:44,997::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.572b0496-5eab-41cd-a976-4514f111898e`ReqID=`1569188a-885b-4665-8cf1-fb18fb2da804`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1198' at 'attachStorageDomain' >Thread-338::DEBUG::2014-05-13 12:42:44,997::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' for lock type 'exclusive' >Thread-338::DEBUG::2014-05-13 12:42:44,997::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' is free. Now locking as 'exclusive' (1 active user) >Thread-338::DEBUG::2014-05-13 12:42:44,998::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.572b0496-5eab-41cd-a976-4514f111898e`ReqID=`1569188a-885b-4665-8cf1-fb18fb2da804`::Granted request >Thread-338::DEBUG::2014-05-13 12:42:44,998::task::827::TaskManager.Task::(resourceAcquired) Task=`1a868a14-4e2d-4257-b6f2-7c938dd7211f`::_resourcesAcquired: Storage.572b0496-5eab-41cd-a976-4514f111898e (exclusive) >Thread-338::DEBUG::2014-05-13 12:42:44,998::task::990::TaskManager.Task::(_decref) Task=`1a868a14-4e2d-4257-b6f2-7c938dd7211f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:42:44,998::sp::869::Storage.StoragePool::(attachSD) sdUUID=572b0496-5eab-41cd-a976-4514f111898e spUUID=aa069361-8e06-4459-a17b-748c0a2a828e >Thread-338::DEBUG::2014-05-13 12:42:44,999::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:42:45,001::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:42:45,059::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:42:45,142::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384762 s, 26.6 MB/s\n'; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:42:45,228::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:42:45,229::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:42:45,230::misc::762::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-338::DEBUG::2014-05-13 12:42:45,230::misc::764::SamplingMethod::(__call__) Got in to sampling method >Thread-338::DEBUG::2014-05-13 12:42:45,230::misc::762::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-338::DEBUG::2014-05-13 12:42:45,230::misc::764::SamplingMethod::(__call__) Got in to sampling method >Thread-338::DEBUG::2014-05-13 12:42:45,230::iscsi::407::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-338::DEBUG::2014-05-13 12:42:45,231::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-338::DEBUG::2014-05-13 12:42:45,253::misc::772::SamplingMethod::(__call__) Returning last result >Thread-338::DEBUG::2014-05-13 12:42:45,254::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >Thread-338::DEBUG::2014-05-13 12:42:45,448::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:42:45,449::lvm::497::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:42:45,450::lvm::499::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:42:45,451::lvm::508::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:42:45,451::lvm::510::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:42:45,452::lvm::528::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:42:45,452::lvm::530::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:42:45,453::misc::772::SamplingMethod::(__call__) Returning last result >Thread-338::DEBUG::2014-05-13 12:42:45,462::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e >Thread-338::DEBUG::2014-05-13 12:42:45,465::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-338::DEBUG::2014-05-13 12:42:45,479::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=NFS_2', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.160.108:/RHEV/ogofen/3', 'ROLE=Regular', 'SDUUID=572b0496-5eab-41cd-a976-4514f111898e', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=06d8db54b521c956db503c23b9077be197d58617'] >Thread-338::DEBUG::2014-05-13 12:42:45,482::fileSD::609::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-338::INFO::2014-05-13 12:42:45,482::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 572b0496-5eab-41cd-a976-4514f111898e_imageNS already registered >Thread-338::INFO::2014-05-13 12:42:45,482::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 572b0496-5eab-41cd-a976-4514f111898e_volumeNS already registered >Thread-338::INFO::2014-05-13 12:42:45,483::clusterlock::184::SANLock::(acquireHostId) Acquiring host id for domain 572b0496-5eab-41cd-a976-4514f111898e (id: 1) >BindingXMLRPC::ERROR::2014-05-13 12:42:45,582::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:42:47,156::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:42:47,252::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0524409 s, 19.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:42:48,603::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:42:49,265::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:42:49,341::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.03232 s, 31.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:42:51,354::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:42:51,435::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363631 s, 28.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:42:51,625::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:42:53,443::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:42:53,520::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0330028 s, 31.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:42:53,856::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:42:53,858::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 12:42:54,064::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:42:54,066::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:42:54,069::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:42:54,080::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000719673 s, 5.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:42:54,644::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:42:55,533::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:42:55,617::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321739 s, 31.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:42:55,970::task::595::TaskManager.Task::(_updateState) Task=`59c958d8-cb9e-41a1-96a7-f4900682baf0`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:42:55,970::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:42:55,970::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000719673', 'lastCheck': '1.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:42:55,971::task::1185::TaskManager.Task::(prepare) Task=`59c958d8-cb9e-41a1-96a7-f4900682baf0`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000719673', 'lastCheck': '1.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:42:55,971::task::595::TaskManager.Task::(_updateState) Task=`59c958d8-cb9e-41a1-96a7-f4900682baf0`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:42:55,971::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:42:55,971::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:42:55,971::task::990::TaskManager.Task::(_decref) Task=`59c958d8-cb9e-41a1-96a7-f4900682baf0`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:42:57,631::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:42:57,663::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:42:57,711::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352652 s, 29.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:42:59,724::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:00,096::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.329181 s, 3.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:43:00,682::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:43:02,109::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:02,189::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382312 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:43:03,701::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:43:04,088::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:43:04,095::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000639202 s, 6.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:43:04,202::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:04,282::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384857 s, 26.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:43:06,295::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:06,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.042377 s, 24.2 MB/s\n'; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:43:06,487::clusterlock::202::SANLock::(acquireHostId) Host id for domain 572b0496-5eab-41cd-a976-4514f111898e successfully acquired (id: 1) >Thread-338::INFO::2014-05-13 12:43:06,492::clusterlock::235::SANLock::(acquire) Acquiring cluster lock for domain 572b0496-5eab-41cd-a976-4514f111898e (id: 1) >Thread-338::DEBUG::2014-05-13 12:43:06,525::clusterlock::261::SANLock::(acquire) Cluster lock for domain 572b0496-5eab-41cd-a976-4514f111898e successfully acquired (id: 1) >Thread-338::DEBUG::2014-05-13 12:43:06,532::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=NFS_2', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.160.108:/RHEV/ogofen/3', 'ROLE=Regular', 'SDUUID=572b0496-5eab-41cd-a976-4514f111898e', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=06d8db54b521c956db503c23b9077be197d58617'] >Thread-338::DEBUG::2014-05-13 12:43:06,532::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-338::DEBUG::2014-05-13 12:43:06,532::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >Thread-338::DEBUG::2014-05-13 12:43:06,532::persistentDict::299::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=NFS_2', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', 'REMOTE_PATH=10.35.160.108:/RHEV/ogofen/3', 'ROLE=Regular', 'SDUUID=572b0496-5eab-41cd-a976-4514f111898e', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f1b43ab3c3813d3f76f0a669539e61d7b52e1053'] >Thread-338::DEBUG::2014-05-13 12:43:06,536::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-338::DEBUG::2014-05-13 12:43:06,536::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-338::DEBUG::2014-05-13 12:43:06,537::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >Thread-338::DEBUG::2014-05-13 12:43:06,537::persistentDict::299::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=['CLASS=Data', 'DESCRIPTION=ISCSI', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'MASTER_VERSION=1', 'PHYBLKSIZE=512', 'POOL_DESCRIPTION=DC_2', 'POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Attached', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', 'ROLE=Master', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'TYPE=ISCSI', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', '_SHA_CKSUM=a158fde071de20e81a8cd49e14bf444b0a25ff23'] >Thread-338::DEBUG::2014-05-13 12:43:06,537::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:06,537::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:06,538::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:06,538::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:06,538::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:06,539::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:43:06,719::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-338::DEBUG::2014-05-13 12:43:06,738::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:43:06,739::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:06,740::blockSD::315::storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT__SHA_CKSUM=a158fde071de20e81a8cd49e14bf444b0a25ff23, MDT_POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active&44&572b0496-5eab-41cd-a976-4514f111898e:Attached removing=MDT_POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active, MDT__SHA_CKSUM=28bb2d7cc60e0d779eb69fe93d050c16c769f069 >Thread-338::DEBUG::2014-05-13 12:43:06,741::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --deltag MDT__SHA_CKSUM=28bb2d7cc60e0d779eb69fe93d050c16c769f069 --deltag MDT_POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active --addtag MDT_POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active&44&572b0496-5eab-41cd-a976-4514f111898e:Attached --addtag MDT__SHA_CKSUM=a158fde071de20e81a8cd49e14bf444b0a25ff23 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-338::DEBUG::2014-05-13 12:43:07,249::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:43:07,250::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:07,250::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:07,251::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-338::INFO::2014-05-13 12:43:07,251::sp::1113::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e to /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/572b0496-5eab-41cd-a976-4514f111898e >Thread-338::INFO::2014-05-13 12:43:07,252::clusterlock::279::SANLock::(release) Releasing cluster lock for domain 572b0496-5eab-41cd-a976-4514f111898e >Thread-338::DEBUG::2014-05-13 12:43:07,254::clusterlock::289::SANLock::(release) Cluster lock for domain 572b0496-5eab-41cd-a976-4514f111898e successfully released >Thread-338::INFO::2014-05-13 12:43:07,254::clusterlock::207::SANLock::(releaseHostId) Releasing host id for domain 572b0496-5eab-41cd-a976-4514f111898e (id: 1) >Dummy-337::DEBUG::2014-05-13 12:43:08,400::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:08,481::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0395256 s, 25.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:43:09,735::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-338::DEBUG::2014-05-13 12:43:10,256::clusterlock::217::SANLock::(releaseHostId) Host id for domain 572b0496-5eab-41cd-a976-4514f111898e released successfully (id: 1) >Thread-338::DEBUG::2014-05-13 12:43:10,256::misc::762::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-338::DEBUG::2014-05-13 12:43:10,256::misc::764::SamplingMethod::(__call__) Got in to sampling method >Thread-338::DEBUG::2014-05-13 12:43:10,257::misc::772::SamplingMethod::(__call__) Returning last result >Thread-338::INFO::2014-05-13 12:43:10,257::logUtils::47::dispatcher::(wrapper) Run and protect: attachStorageDomain, Return response: None >Thread-338::DEBUG::2014-05-13 12:43:10,257::task::1185::TaskManager.Task::(prepare) Task=`1a868a14-4e2d-4257-b6f2-7c938dd7211f`::finished: None >Thread-338::DEBUG::2014-05-13 12:43:10,257::task::595::TaskManager.Task::(_updateState) Task=`1a868a14-4e2d-4257-b6f2-7c938dd7211f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:43:10,257::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.572b0496-5eab-41cd-a976-4514f111898e': < ResourceRef 'Storage.572b0496-5eab-41cd-a976-4514f111898e', isValid: 'True' obj: 'None'>, 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:43:10,258::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:43:10,258::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' >Thread-338::DEBUG::2014-05-13 12:43:10,258::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:43:10,258::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:43:10,258::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:43:10,259::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:43:10,259::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:43:10,259::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:43:10,259::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:43:10,259::task::990::TaskManager.Task::(_decref) Task=`1a868a14-4e2d-4257-b6f2-7c938dd7211f`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:43:10,273::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:43:10,274::task::595::TaskManager.Task::(_updateState) Task=`99dcaa95-f5e3-4d1a-9cbb-97c079103b69`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:43:10,274::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:43:10,274::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:43:10,274::task::1185::TaskManager.Task::(prepare) Task=`99dcaa95-f5e3-4d1a-9cbb-97c079103b69`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:43:10,274::task::595::TaskManager.Task::(_updateState) Task=`99dcaa95-f5e3-4d1a-9cbb-97c079103b69`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:43:10,274::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:43:10,275::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:43:10,275::task::990::TaskManager.Task::(_decref) Task=`99dcaa95-f5e3-4d1a-9cbb-97c079103b69`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:43:10,279::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:43:10,279::task::595::TaskManager.Task::(_updateState) Task=`b6baebf7-8fb2-4e47-b352-7058ea7b1eeb`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:43:10,279::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:43:10,280::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`88c598d4-802f-434c-b26c-aec4f23ccd12`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:43:10,280::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:43:10,280::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:43:10,280::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`88c598d4-802f-434c-b26c-aec4f23ccd12`::Granted request >Thread-338::DEBUG::2014-05-13 12:43:10,281::task::827::TaskManager.Task::(resourceAcquired) Task=`b6baebf7-8fb2-4e47-b352-7058ea7b1eeb`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:43:10,281::task::990::TaskManager.Task::(_decref) Task=`b6baebf7-8fb2-4e47-b352-7058ea7b1eeb`::ref 1 aborting False >Thread-338::DEBUG::2014-05-13 12:43:10,281::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:10,282::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-66::DEBUG::2014-05-13 12:43:10,307::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5dd29152] >Thread-66::DEBUG::2014-05-13 12:43:10,307::task::595::TaskManager.Task::(_updateState) Task=`ad07da99-368c-4995-964b-bf069021a4eb`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:43:10,308::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.160.108:/RHEV/ogofen/3', 'iqn': '', 'user': '', 'tpgt': '1', 'password': '******', 'id': '3a6bb180-87f9-441b-a52a-530eb2e61642', 'port': ''}], options=None) >Thread-66::DEBUG::2014-05-13 12:43:10,310::hsm::2328::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3 >Thread-66::DEBUG::2014-05-13 12:43:10,312::hsm::2352::Storage.HSM::(__prefetchDomains) Found SD uuids: ('572b0496-5eab-41cd-a976-4514f111898e',) >Thread-66::DEBUG::2014-05-13 12:43:10,312::hsm::2408::Storage.HSM::(connectStorageServer) knownSDs: {1ee9ca05-ac9d-4d53-ac4b-6afbe1da0850: storage.nfsSD.findDomain, 1bc6ed6b-8167-4de8-bc1e-ed625a0274d3: storage.blockSD.findDomain, 49db7ad8-2283-4caf-ad27-5dac5b9cb950: storage.blockSD.findDomain, 2874d6f5-ac53-466a-973b-04ddc3beaa24: storage.blockSD.findDomain, 3e8d0026-71e2-43b8-a6cd-853b8861201b: storage.nfsSD.findDomain, 689a6ef7-211a-4bcb-8ac1-42f5cc467967: storage.nfsSD.findDomain, d5af43fc-0c38-4d75-89ad-216daa092503: storage.nfsSD.findDomain, 598916de-8a64-4485-bccb-bc3cbb623569: storage.nfsSD.findDomain, f853f6fe-5b39-4f03-b8a5-dad5823f0bb7: storage.blockSD.findDomain, bfd13673-a430-442d-beb5-dd42d2107b7d: storage.nfsSD.findDomain, 572b0496-5eab-41cd-a976-4514f111898e: storage.nfsSD.findDomain, 23e7e31a-283a-4398-920a-b3ba8038cb9f: storage.blockSD.findDomain, e4894e28-76eb-4855-bbd1-580c11d4ad51: storage.nfsSD.findDomain, 41830dad-f012-4b4d-b636-863f66d5ca4b: storage.blockSD.findDomain, 6be03a71-472c-4714-bf93-340fdf604955: storage.blockSD.findDomain, c7b37830-fa22-4e63-a8e6-ed70e742a188: storage.nfsSD.findDomain, 9f4c0902-8dba-44c3-8181-9a6b649c61ff: storage.nfsSD.findDomain} >Thread-66::INFO::2014-05-13 12:43:10,312::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '3a6bb180-87f9-441b-a52a-530eb2e61642'}]} >Thread-66::DEBUG::2014-05-13 12:43:10,313::task::1185::TaskManager.Task::(prepare) Task=`ad07da99-368c-4995-964b-bf069021a4eb`::finished: {'statuslist': [{'status': 0, 'id': '3a6bb180-87f9-441b-a52a-530eb2e61642'}]} >Thread-66::DEBUG::2014-05-13 12:43:10,313::task::595::TaskManager.Task::(_updateState) Task=`ad07da99-368c-4995-964b-bf069021a4eb`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:43:10,313::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:43:10,313::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:43:10,313::task::990::TaskManager.Task::(_decref) Task=`ad07da99-368c-4995-964b-bf069021a4eb`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:43:10,494::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-338::DEBUG::2014-05-13 12:43:10,518::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:43:10,522::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-338::INFO::2014-05-13 12:43:10,526::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Attached', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, '572b0496-5eab-41cd-a976-4514f111898e': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-338::DEBUG::2014-05-13 12:43:10,527::task::1185::TaskManager.Task::(prepare) Task=`b6baebf7-8fb2-4e47-b352-7058ea7b1eeb`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Attached', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, '572b0496-5eab-41cd-a976-4514f111898e': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-338::DEBUG::2014-05-13 12:43:10,528::task::595::TaskManager.Task::(_updateState) Task=`b6baebf7-8fb2-4e47-b352-7058ea7b1eeb`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:43:10,529::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:43:10,530::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:43:10,531::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:43:10,532::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:43:10,533::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:43:10,534::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:43:10,535::task::990::TaskManager.Task::(_decref) Task=`b6baebf7-8fb2-4e47-b352-7058ea7b1eeb`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:43:10,557::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5fc30331] >Thread-338::DEBUG::2014-05-13 12:43:10,559::task::595::TaskManager.Task::(_updateState) Task=`f59ec744-0af4-4774-a988-fc529bf930ab`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:43:10,560::logUtils::44::dispatcher::(wrapper) Run and protect: activateStorageDomain(sdUUID='572b0496-5eab-41cd-a976-4514f111898e', spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:43:10,562::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9dab2906-060a-4852-b66e-2f04aba2eaa7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1256' at 'activateStorageDomain' >Thread-338::DEBUG::2014-05-13 12:43:10,563::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'exclusive' >Thread-338::DEBUG::2014-05-13 12:43:10,564::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'exclusive' (1 active user) >Thread-338::DEBUG::2014-05-13 12:43:10,565::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9dab2906-060a-4852-b66e-2f04aba2eaa7`::Granted request >Thread-338::DEBUG::2014-05-13 12:43:10,567::task::827::TaskManager.Task::(resourceAcquired) Task=`f59ec744-0af4-4774-a988-fc529bf930ab`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (exclusive) >Thread-338::DEBUG::2014-05-13 12:43:10,568::task::990::TaskManager.Task::(_decref) Task=`f59ec744-0af4-4774-a988-fc529bf930ab`::ref 1 aborting False >Thread-338::DEBUG::2014-05-13 12:43:10,570::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.572b0496-5eab-41cd-a976-4514f111898e`ReqID=`0d8ec399-6e2e-4bdb-9b4f-b3d7c986de34`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1257' at 'activateStorageDomain' >Thread-338::DEBUG::2014-05-13 12:43:10,571::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' for lock type 'exclusive' >Thread-338::DEBUG::2014-05-13 12:43:10,573::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' is free. Now locking as 'exclusive' (1 active user) >Thread-338::DEBUG::2014-05-13 12:43:10,574::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.572b0496-5eab-41cd-a976-4514f111898e`ReqID=`0d8ec399-6e2e-4bdb-9b4f-b3d7c986de34`::Granted request >Thread-338::DEBUG::2014-05-13 12:43:10,576::task::827::TaskManager.Task::(resourceAcquired) Task=`f59ec744-0af4-4774-a988-fc529bf930ab`::_resourcesAcquired: Storage.572b0496-5eab-41cd-a976-4514f111898e (exclusive) >Thread-338::DEBUG::2014-05-13 12:43:10,577::task::990::TaskManager.Task::(_decref) Task=`f59ec744-0af4-4774-a988-fc529bf930ab`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:43:10,578::sp::1024::Storage.StoragePool::(activateSD) sdUUID=572b0496-5eab-41cd-a976-4514f111898e spUUID=aa069361-8e06-4459-a17b-748c0a2a828e >Thread-338::DEBUG::2014-05-13 12:43:10,578::misc::762::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-338::DEBUG::2014-05-13 12:43:10,579::misc::764::SamplingMethod::(__call__) Got in to sampling method >Thread-338::DEBUG::2014-05-13 12:43:10,580::misc::762::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-338::DEBUG::2014-05-13 12:43:10,581::misc::764::SamplingMethod::(__call__) Got in to sampling method >Thread-338::DEBUG::2014-05-13 12:43:10,582::iscsi::407::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-338::DEBUG::2014-05-13 12:43:10,583::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:10,630::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0819241 s, 12.5 MB/s\n'; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:43:10,633::misc::772::SamplingMethod::(__call__) Returning last result >Thread-338::DEBUG::2014-05-13 12:43:10,635::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >Thread-338::DEBUG::2014-05-13 12:43:10,944::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:43:10,945::lvm::497::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:10,945::lvm::499::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:10,946::lvm::508::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:10,946::lvm::510::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:10,946::lvm::528::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:10,947::lvm::530::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:10,947::misc::772::SamplingMethod::(__call__) Returning last result >Thread-338::DEBUG::2014-05-13 12:43:10,953::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e >Thread-338::DEBUG::2014-05-13 12:43:10,955::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-338::DEBUG::2014-05-13 12:43:10,965::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=NFS_2', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', 'REMOTE_PATH=10.35.160.108:/RHEV/ogofen/3', 'ROLE=Regular', 'SDUUID=572b0496-5eab-41cd-a976-4514f111898e', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f1b43ab3c3813d3f76f0a669539e61d7b52e1053'] >Thread-338::DEBUG::2014-05-13 12:43:10,968::fileSD::609::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-338::INFO::2014-05-13 12:43:10,968::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 572b0496-5eab-41cd-a976-4514f111898e_imageNS already registered >Thread-338::INFO::2014-05-13 12:43:10,969::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 572b0496-5eab-41cd-a976-4514f111898e_volumeNS already registered >Thread-338::INFO::2014-05-13 12:43:10,970::sp::1113::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e to /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/572b0496-5eab-41cd-a976-4514f111898e >Thread-338::DEBUG::2014-05-13 12:43:10,971::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-338::DEBUG::2014-05-13 12:43:10,971::persistentDict::160::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-338::DEBUG::2014-05-13 12:43:10,971::persistentDict::160::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-338::DEBUG::2014-05-13 12:43:10,972::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >Thread-338::DEBUG::2014-05-13 12:43:10,972::persistentDict::299::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=NFS_2', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', 'REMOTE_PATH=10.35.160.108:/RHEV/ogofen/3', 'ROLE=Regular', 'SDUUID=572b0496-5eab-41cd-a976-4514f111898e', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=06893a9d3c8506053131030a281fb9b9deb4efa9'] >Thread-338::DEBUG::2014-05-13 12:43:10,982::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-338::DEBUG::2014-05-13 12:43:10,986::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e >Thread-338::DEBUG::2014-05-13 12:43:10,987::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-338::DEBUG::2014-05-13 12:43:10,994::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=NFS_2', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', 'REMOTE_PATH=10.35.160.108:/RHEV/ogofen/3', 'ROLE=Regular', 'SDUUID=572b0496-5eab-41cd-a976-4514f111898e', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=06893a9d3c8506053131030a281fb9b9deb4efa9'] >Thread-338::DEBUG::2014-05-13 12:43:10,995::fileSD::609::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-338::INFO::2014-05-13 12:43:10,995::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 572b0496-5eab-41cd-a976-4514f111898e_imageNS already registered >Thread-338::INFO::2014-05-13 12:43:10,996::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 572b0496-5eab-41cd-a976-4514f111898e_volumeNS already registered >Thread-338::DEBUG::2014-05-13 12:43:10,996::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-338::DEBUG::2014-05-13 12:43:10,996::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >Thread-338::DEBUG::2014-05-13 12:43:10,997::persistentDict::299::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=['CLASS=Data', 'DESCRIPTION=ISCSI', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'MASTER_VERSION=1', 'PHYBLKSIZE=512', 'POOL_DESCRIPTION=DC_2', 'POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', 'ROLE=Master', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'TYPE=ISCSI', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', '_SHA_CKSUM=755a5160f5b8caaab19e905b201f8ce3ebcf5fb0'] >Thread-338::DEBUG::2014-05-13 12:43:10,997::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:10,997::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:10,997::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:10,998::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:10,998::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:11,000::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-338::DEBUG::2014-05-13 12:43:11,272::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:43:11,273::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:11,274::blockSD::315::storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT_POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active&44&572b0496-5eab-41cd-a976-4514f111898e:Active, MDT__SHA_CKSUM=755a5160f5b8caaab19e905b201f8ce3ebcf5fb0 removing=MDT__SHA_CKSUM=a158fde071de20e81a8cd49e14bf444b0a25ff23, MDT_POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active&44&572b0496-5eab-41cd-a976-4514f111898e:Attached >Thread-338::DEBUG::2014-05-13 12:43:11,275::lvm::296::Storage.Misc.excCmd::(cmd) u'/usr/bin/sudo -n /sbin/lvm vgchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --deltag MDT_POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active&44&572b0496-5eab-41cd-a976-4514f111898e:Attached --deltag MDT__SHA_CKSUM=a158fde071de20e81a8cd49e14bf444b0a25ff23 --addtag MDT__SHA_CKSUM=755a5160f5b8caaab19e905b201f8ce3ebcf5fb0 --addtag MDT_POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active&44&572b0496-5eab-41cd-a976-4514f111898e:Active 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-66::DEBUG::2014-05-13 12:43:11,435::task::595::TaskManager.Task::(_updateState) Task=`2c88440a-9252-418b-a613-9254ad1c5a49`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:43:11,435::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:43:11,436::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000639202', 'lastCheck': '7.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:43:11,436::task::1185::TaskManager.Task::(prepare) Task=`2c88440a-9252-418b-a613-9254ad1c5a49`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000639202', 'lastCheck': '7.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:43:11,436::task::595::TaskManager.Task::(_updateState) Task=`2c88440a-9252-418b-a613-9254ad1c5a49`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:43:11,436::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:43:11,436::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:43:11,437::task::990::TaskManager.Task::(_decref) Task=`2c88440a-9252-418b-a613-9254ad1c5a49`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:43:11,687::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:43:11,687::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:11,688::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:11,688::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-338::DEBUG::2014-05-13 12:43:11,688::misc::762::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-338::DEBUG::2014-05-13 12:43:11,689::misc::764::SamplingMethod::(__call__) Got in to sampling method >Thread-338::INFO::2014-05-13 12:43:11,689::domainMonitor::99::Storage.DomainMonitor::(startMonitoring) Start monitoring 572b0496-5eab-41cd-a976-4514f111898e >Thread-344::DEBUG::2014-05-13 12:43:11,690::domainMonitor::165::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for 572b0496-5eab-41cd-a976-4514f111898e >Thread-338::DEBUG::2014-05-13 12:43:11,690::sp::1404::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `aa069361-8e06-4459-a17b-748c0a2a828e` started monitoring domain `572b0496-5eab-41cd-a976-4514f111898e` >Thread-338::DEBUG::2014-05-13 12:43:11,691::misc::772::SamplingMethod::(__call__) Returning last result >Thread-338::INFO::2014-05-13 12:43:11,693::logUtils::47::dispatcher::(wrapper) Run and protect: activateStorageDomain, Return response: None >Thread-338::DEBUG::2014-05-13 12:43:11,694::task::1185::TaskManager.Task::(prepare) Task=`f59ec744-0af4-4774-a988-fc529bf930ab`::finished: None >Thread-338::DEBUG::2014-05-13 12:43:11,695::task::595::TaskManager.Task::(_updateState) Task=`f59ec744-0af4-4774-a988-fc529bf930ab`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:43:11,696::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.572b0496-5eab-41cd-a976-4514f111898e': < ResourceRef 'Storage.572b0496-5eab-41cd-a976-4514f111898e', isValid: 'True' obj: 'None'>, 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:43:11,698::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:43:11,701::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' >Thread-338::DEBUG::2014-05-13 12:43:11,703::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:43:11,705::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:43:11,706::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:43:11,708::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:43:11,708::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:43:11,708::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:43:11,709::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:43:11,710::task::990::TaskManager.Task::(_decref) Task=`f59ec744-0af4-4774-a988-fc529bf930ab`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:43:11,717::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:43:11,726::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000471965 s, 737 kB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:43:11,728::domainMonitor::247::Storage.DomainMonitorThread::(_monitorDomain) Domain 572b0496-5eab-41cd-a976-4514f111898e changed its status to Valid >Thread-345::DEBUG::2014-05-13 12:43:11,728::misc::884::Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Emitting event >Thread-344::INFO::2014-05-13 12:43:11,729::clusterlock::184::SANLock::(acquireHostId) Acquiring host id for domain 572b0496-5eab-41cd-a976-4514f111898e (id: 1) >Thread-345::DEBUG::2014-05-13 12:43:11,730::misc::894::Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `_onDomainStateChange` >Thread-344::DEBUG::2014-05-13 12:43:11,730::clusterlock::202::SANLock::(acquireHostId) Host id for domain 572b0496-5eab-41cd-a976-4514f111898e successfully acquired (id: 1) >Thread-345::DEBUG::2014-05-13 12:43:11,731::misc::894::Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `contEIOVms` >Thread-346::INFO::2014-05-13 12:43:11,733::sp::1113::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e to /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/572b0496-5eab-41cd-a976-4514f111898e >Thread-345::DEBUG::2014-05-13 12:43:11,733::misc::894::Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `_upgradePoolDomain` >Thread-345::DEBUG::2014-05-13 12:43:11,735::misc::904::Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Event emitted >Thread-347::INFO::2014-05-13 12:43:11,741::clientIF::126::vds::(contEIOVms) vmContainerLock acquired >Thread-66::DEBUG::2014-05-13 12:43:11,764::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5fc30331] >Thread-66::DEBUG::2014-05-13 12:43:11,765::task::595::TaskManager.Task::(_updateState) Task=`92fd4dcb-1f43-4223-8df5-5a572f7887f2`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:43:11,765::logUtils::44::dispatcher::(wrapper) Run and protect: refreshStoragePool(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', msdUUID='41830dad-f012-4b4d-b636-863f66d5ca4b', masterVersion=1, options=None) >Thread-66::DEBUG::2014-05-13 12:43:11,765::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e257a436-cdf9-45c3-be6a-c4c1a554cd94`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '870' at 'refreshStoragePool' >Thread-66::DEBUG::2014-05-13 12:43:11,765::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-66::DEBUG::2014-05-13 12:43:11,766::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-66::DEBUG::2014-05-13 12:43:11,766::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e257a436-cdf9-45c3-be6a-c4c1a554cd94`::Granted request >Thread-66::DEBUG::2014-05-13 12:43:11,766::task::827::TaskManager.Task::(resourceAcquired) Task=`92fd4dcb-1f43-4223-8df5-5a572f7887f2`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-66::DEBUG::2014-05-13 12:43:11,766::task::990::TaskManager.Task::(_decref) Task=`92fd4dcb-1f43-4223-8df5-5a572f7887f2`::ref 1 aborting False >Thread-66::INFO::2014-05-13 12:43:11,767::hsm::879::Storage.HSM::(refreshStoragePool) Ignoring the refreshStoragePool request (the host is the SPM) >Thread-66::INFO::2014-05-13 12:43:11,767::logUtils::47::dispatcher::(wrapper) Run and protect: refreshStoragePool, Return response: None >Thread-66::DEBUG::2014-05-13 12:43:11,767::task::1185::TaskManager.Task::(prepare) Task=`92fd4dcb-1f43-4223-8df5-5a572f7887f2`::finished: None >Thread-66::DEBUG::2014-05-13 12:43:11,767::task::595::TaskManager.Task::(_updateState) Task=`92fd4dcb-1f43-4223-8df5-5a572f7887f2`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:43:11,767::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-66::DEBUG::2014-05-13 12:43:11,767::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:43:11,768::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-66::DEBUG::2014-05-13 12:43:11,768::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-66::DEBUG::2014-05-13 12:43:11,768::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-66::DEBUG::2014-05-13 12:43:11,768::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-66::DEBUG::2014-05-13 12:43:11,768::task::990::TaskManager.Task::(_decref) Task=`92fd4dcb-1f43-4223-8df5-5a572f7887f2`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:43:12,654::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:43:12,755::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:43:13,024::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.327852 s, 3.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:43:14,102::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:43:14,103::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 12:43:14,319::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:43:14,320::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:43:14,324::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:43:14,332::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000683787 s, 6.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:43:15,036::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:15,117::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0394504 s, 26.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:43:15,778::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:43:17,130::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:17,214::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0392736 s, 26.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:43:18,797::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:43:19,227::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:19,301::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312484 s, 32.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:43:20,559::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:43:20,560::task::595::TaskManager.Task::(_updateState) Task=`654a4542-f3e7-4375-9673-0f45c78ec54e`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:43:20,560::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:43:20,560::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:43:20,560::task::1185::TaskManager.Task::(prepare) Task=`654a4542-f3e7-4375-9673-0f45c78ec54e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:43:20,561::task::595::TaskManager.Task::(_updateState) Task=`654a4542-f3e7-4375-9673-0f45c78ec54e`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:43:20,561::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:43:20,561::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:43:20,561::task::990::TaskManager.Task::(_decref) Task=`654a4542-f3e7-4375-9673-0f45c78ec54e`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:43:20,605::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:43:20,606::task::595::TaskManager.Task::(_updateState) Task=`2d921799-04bc-495a-a0d6-61d4ca0f3273`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:43:20,606::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:43:20,606::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d03e3d1c-e1e0-49a7-9894-ea5a5e13b1c8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:43:20,606::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:43:20,607::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:43:20,607::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d03e3d1c-e1e0-49a7-9894-ea5a5e13b1c8`::Granted request >Thread-338::DEBUG::2014-05-13 12:43:20,607::task::827::TaskManager.Task::(resourceAcquired) Task=`2d921799-04bc-495a-a0d6-61d4ca0f3273`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:43:20,608::task::990::TaskManager.Task::(_decref) Task=`2d921799-04bc-495a-a0d6-61d4ca0f3273`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:43:20,608::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, '572b0496-5eab-41cd-a976-4514f111898e': {'status': 'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:43:20,609::task::1185::TaskManager.Task::(prepare) Task=`2d921799-04bc-495a-a0d6-61d4ca0f3273`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, '572b0496-5eab-41cd-a976-4514f111898e': {'status': 'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:43:20,609::task::595::TaskManager.Task::(_updateState) Task=`2d921799-04bc-495a-a0d6-61d4ca0f3273`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:43:20,609::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:43:20,609::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:43:20,609::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:43:20,610::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:43:20,610::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:43:20,610::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:43:20,610::task::990::TaskManager.Task::(_decref) Task=`2d921799-04bc-495a-a0d6-61d4ca0f3273`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:43:21,314::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:21,434::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0523318 s, 19.6 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:43:21,742::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:43:21,751::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000330239 s, 1.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:43:21,816::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:43:23,453::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:23,875::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.378448 s, 2.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:43:24,340::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:43:24,348::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000662839 s, 6.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:43:24,839::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:43:25,888::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:25,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320304 s, 32.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:43:26,566::task::595::TaskManager.Task::(_updateState) Task=`161c1b30-8d62-4af1-b5d7-7dd884ee62eb`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:43:26,566::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:43:26,567::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000662839', 'lastCheck': '2.2', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000330239', 'lastCheck': '4.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:43:26,567::task::1185::TaskManager.Task::(prepare) Task=`161c1b30-8d62-4af1-b5d7-7dd884ee62eb`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000662839', 'lastCheck': '2.2', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000330239', 'lastCheck': '4.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:43:26,567::task::595::TaskManager.Task::(_updateState) Task=`161c1b30-8d62-4af1-b5d7-7dd884ee62eb`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:43:26,567::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:43:26,567::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:43:26,567::task::990::TaskManager.Task::(_decref) Task=`161c1b30-8d62-4af1-b5d7-7dd884ee62eb`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:43:27,860::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:43:27,977::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:28,054::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321993 s, 31.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:43:30,068::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:30,151::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0386818 s, 26.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:43:30,630::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:43:30,631::task::595::TaskManager.Task::(_updateState) Task=`90d0829a-6d87-45ad-a37b-21abf06e34fe`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:43:30,631::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:43:30,631::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:43:30,631::task::1185::TaskManager.Task::(prepare) Task=`90d0829a-6d87-45ad-a37b-21abf06e34fe`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:43:30,632::task::595::TaskManager.Task::(_updateState) Task=`90d0829a-6d87-45ad-a37b-21abf06e34fe`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:43:30,632::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:43:30,632::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:43:30,632::task::990::TaskManager.Task::(_decref) Task=`90d0829a-6d87-45ad-a37b-21abf06e34fe`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:43:30,637::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:43:30,637::task::595::TaskManager.Task::(_updateState) Task=`71f66b2a-71b0-43fe-aade-cb43d8369917`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:43:30,638::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:43:30,638::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5d262876-5438-4ec0-b780-489cfec62210`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:43:30,638::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:43:30,638::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:43:30,638::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`5d262876-5438-4ec0-b780-489cfec62210`::Granted request >Thread-338::DEBUG::2014-05-13 12:43:30,639::task::827::TaskManager.Task::(resourceAcquired) Task=`71f66b2a-71b0-43fe-aade-cb43d8369917`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:43:30,639::task::990::TaskManager.Task::(_decref) Task=`71f66b2a-71b0-43fe-aade-cb43d8369917`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:43:30,640::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, '572b0496-5eab-41cd-a976-4514f111898e': {'status': 'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:43:30,640::task::1185::TaskManager.Task::(prepare) Task=`71f66b2a-71b0-43fe-aade-cb43d8369917`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': '41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': 'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, '572b0496-5eab-41cd-a976-4514f111898e': {'status': 'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:43:30,640::task::595::TaskManager.Task::(_updateState) Task=`71f66b2a-71b0-43fe-aade-cb43d8369917`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:43:30,640::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:43:30,640::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:43:30,641::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:43:30,641::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:43:30,641::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:43:30,641::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:43:30,641::task::990::TaskManager.Task::(_decref) Task=`71f66b2a-71b0-43fe-aade-cb43d8369917`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:43:30,879::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-344::DEBUG::2014-05-13 12:43:31,762::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:43:31,770::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000354644 s, 981 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:43:32,158::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [18ef8f20] >Thread-66::DEBUG::2014-05-13 12:43:32,159::task::595::TaskManager.Task::(_updateState) Task=`239d32ec-b167-404a-a778-484d90df1a58`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:43:32,159::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.160.108:/RHEV/ogofen/4', 'iqn': '', 'user': '', 'tpgt': '1', 'password': '******', 'id': '742883c1-8e2a-47ca-8cac-a4ef313efb7f', 'port': ''}], options=None) >Thread-66::DEBUG::2014-05-13 12:43:32,162::mount::226::Storage.Misc.excCmd::(_runcmd) '/usr/bin/sudo -n /bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6,nfsvers=3 10.35.160.108:/RHEV/ogofen/4 /rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_4' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:32,170::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 12:43:32,238::hsm::2328::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_4 >Thread-66::DEBUG::2014-05-13 12:43:32,248::hsm::2352::Storage.HSM::(__prefetchDomains) Found SD uuids: ('689a6ef7-211a-4bcb-8ac1-42f5cc467967',) >Thread-66::DEBUG::2014-05-13 12:43:32,250::hsm::2408::Storage.HSM::(connectStorageServer) knownSDs: {1ee9ca05-ac9d-4d53-ac4b-6afbe1da0850: storage.nfsSD.findDomain, 1bc6ed6b-8167-4de8-bc1e-ed625a0274d3: storage.blockSD.findDomain, 49db7ad8-2283-4caf-ad27-5dac5b9cb950: storage.blockSD.findDomain, 2874d6f5-ac53-466a-973b-04ddc3beaa24: storage.blockSD.findDomain, 3e8d0026-71e2-43b8-a6cd-853b8861201b: storage.nfsSD.findDomain, 689a6ef7-211a-4bcb-8ac1-42f5cc467967: storage.nfsSD.findDomain, d5af43fc-0c38-4d75-89ad-216daa092503: storage.nfsSD.findDomain, 598916de-8a64-4485-bccb-bc3cbb623569: storage.nfsSD.findDomain, f853f6fe-5b39-4f03-b8a5-dad5823f0bb7: storage.blockSD.findDomain, bfd13673-a430-442d-beb5-dd42d2107b7d: storage.nfsSD.findDomain, 572b0496-5eab-41cd-a976-4514f111898e: storage.nfsSD.findDomain, 23e7e31a-283a-4398-920a-b3ba8038cb9f: storage.blockSD.findDomain, e4894e28-76eb-4855-bbd1-580c11d4ad51: storage.nfsSD.findDomain, 41830dad-f012-4b4d-b636-863f66d5ca4b: storage.blockSD.findDomain, 6be03a71-472c-4714-bf93-340fdf604955: storage.blockSD.findDomain, c7b37830-fa22-4e63-a8e6-ed70e742a188: storage.nfsSD.findDomain, 9f4c0902-8dba-44c3-8181-9a6b649c61ff: storage.nfsSD.findDomain} >Thread-66::INFO::2014-05-13 12:43:32,250::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '742883c1-8e2a-47ca-8cac-a4ef313efb7f'}]} >Thread-66::DEBUG::2014-05-13 12:43:32,256::task::1185::TaskManager.Task::(prepare) Task=`239d32ec-b167-404a-a778-484d90df1a58`::finished: {'statuslist': [{'status': 0, 'id': '742883c1-8e2a-47ca-8cac-a4ef313efb7f'}]} >Thread-66::DEBUG::2014-05-13 12:43:32,257::task::595::TaskManager.Task::(_updateState) Task=`239d32ec-b167-404a-a778-484d90df1a58`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:43:32,258::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:43:32,259::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:43:32,260::task::990::TaskManager.Task::(_decref) Task=`239d32ec-b167-404a-a778-484d90df1a58`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:43:32,272::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [50528629] >Thread-338::DEBUG::2014-05-13 12:43:32,274::task::595::TaskManager.Task::(_updateState) Task=`438ca0f8-a0ad-4b41-9ff0-4e06d7e3432a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:43:32,276::logUtils::44::dispatcher::(wrapper) Run and protect: attachStorageDomain(sdUUID='689a6ef7-211a-4bcb-8ac1-42f5cc467967', spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:43:32,279::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`306d396e-a786-4173-8efc-e7e818b47c80`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1197' at 'attachStorageDomain' >Thread-338::DEBUG::2014-05-13 12:43:32,280::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'exclusive' >Thread-338::DEBUG::2014-05-13 12:43:32,282::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'exclusive' (1 active user) >Thread-338::DEBUG::2014-05-13 12:43:32,283::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`306d396e-a786-4173-8efc-e7e818b47c80`::Granted request >Thread-338::DEBUG::2014-05-13 12:43:32,286::task::827::TaskManager.Task::(resourceAcquired) Task=`438ca0f8-a0ad-4b41-9ff0-4e06d7e3432a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (exclusive) >Thread-338::DEBUG::2014-05-13 12:43:32,287::task::990::TaskManager.Task::(_decref) Task=`438ca0f8-a0ad-4b41-9ff0-4e06d7e3432a`::ref 1 aborting False >Thread-338::DEBUG::2014-05-13 12:43:32,290::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.689a6ef7-211a-4bcb-8ac1-42f5cc467967`ReqID=`1a12fb47-ef5a-49fe-b903-7d3ce952901c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1198' at 'attachStorageDomain' >Thread-338::DEBUG::2014-05-13 12:43:32,291::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.689a6ef7-211a-4bcb-8ac1-42f5cc467967' for lock type 'exclusive' >Thread-338::DEBUG::2014-05-13 12:43:32,293::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.689a6ef7-211a-4bcb-8ac1-42f5cc467967' is free. Now locking as 'exclusive' (1 active user) >Thread-338::DEBUG::2014-05-13 12:43:32,294::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.689a6ef7-211a-4bcb-8ac1-42f5cc467967`ReqID=`1a12fb47-ef5a-49fe-b903-7d3ce952901c`::Granted request >Thread-338::DEBUG::2014-05-13 12:43:32,298::task::827::TaskManager.Task::(resourceAcquired) Task=`438ca0f8-a0ad-4b41-9ff0-4e06d7e3432a`::_resourcesAcquired: Storage.689a6ef7-211a-4bcb-8ac1-42f5cc467967 (exclusive) >Thread-338::DEBUG::2014-05-13 12:43:32,299::task::990::TaskManager.Task::(_decref) Task=`438ca0f8-a0ad-4b41-9ff0-4e06d7e3432a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:43:32,301::sp::869::Storage.StoragePool::(attachSD) sdUUID=689a6ef7-211a-4bcb-8ac1-42f5cc467967 spUUID=aa069361-8e06-4459-a17b-748c0a2a828e >Thread-338::DEBUG::2014-05-13 12:43:32,304::misc::762::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-338::DEBUG::2014-05-13 12:43:32,305::misc::764::SamplingMethod::(__call__) Got in to sampling method >Thread-338::DEBUG::2014-05-13 12:43:32,306::misc::762::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-338::DEBUG::2014-05-13 12:43:32,308::misc::764::SamplingMethod::(__call__) Got in to sampling method >Thread-338::DEBUG::2014-05-13 12:43:32,309::iscsi::407::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-338::DEBUG::2014-05-13 12:43:32,311::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-338::DEBUG::2014-05-13 12:43:32,336::misc::772::SamplingMethod::(__call__) Returning last result >Thread-338::DEBUG::2014-05-13 12:43:32,338::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:32,348::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.060298 s, 17.0 MB/s\n'; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:43:32,610::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:43:32,610::lvm::497::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:32,611::lvm::499::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:32,612::lvm::508::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:32,612::lvm::510::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:32,613::lvm::528::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:32,613::lvm::530::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:43:32,614::misc::772::SamplingMethod::(__call__) Returning last result >Thread-338::DEBUG::2014-05-13 12:43:32,628::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_4/689a6ef7-211a-4bcb-8ac1-42f5cc467967 >Thread-338::DEBUG::2014-05-13 12:43:32,630::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-338::DEBUG::2014-05-13 12:43:32,646::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=NFS_1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_DESCRIPTION=DC_1', 'POOL_DOMAINS=1bc6ed6b-8167-4de8-bc1e-ed625a0274d3:Active,689a6ef7-211a-4bcb-8ac1-42f5cc467967:Active,c7b37830-fa22-4e63-a8e6-ed70e742a188:Active', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=1', 'POOL_UUID=', 'REMOTE_PATH=10.35.160.108:/RHEV/ogofen/4', 'ROLE=Regular', 'SDUUID=689a6ef7-211a-4bcb-8ac1-42f5cc467967', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=569b82bacedbb1ce198c7a2dba7e5eeda62a93a9'] >Thread-338::DEBUG::2014-05-13 12:43:32,649::fileSD::609::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-338::INFO::2014-05-13 12:43:32,649::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 689a6ef7-211a-4bcb-8ac1-42f5cc467967_imageNS already registered >Thread-338::INFO::2014-05-13 12:43:32,649::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 689a6ef7-211a-4bcb-8ac1-42f5cc467967_volumeNS already registered >Thread-338::INFO::2014-05-13 12:43:32,650::clusterlock::184::SANLock::(acquireHostId) Acquiring host id for domain 689a6ef7-211a-4bcb-8ac1-42f5cc467967 (id: 1) >BindingXMLRPC::ERROR::2014-05-13 12:43:33,898::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:43:34,353::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:43:34,355::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:34,364::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:34,441::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0328767 s, 31.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:43:34,654::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:43:34,656::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:43:34,748::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:43:34,756::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000723408 s, 5.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:43:36,455::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:36,546::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0468279 s, 21.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:43:36,917::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:43:38,565::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:38,646::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0367472 s, 27.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:43:39,936::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:43:40,515::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [bdab577] >Thread-66::DEBUG::2014-05-13 12:43:40,515::task::595::TaskManager.Task::(_updateState) Task=`97fe9167-8a2b-4dc7-bbb3-e30c212536ba`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:43:40,515::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': 'dhcp163-66.scl.lab.tlv.redhat.com:/var/lib/exports/iso-20140512085215', 'iqn': '', 'user': '', 'tpgt': '', 'password': '******', 'id': '036b03a4-bbab-4c8a-b584-2557b381995a', 'port': ''}], options=None) >Thread-66::DEBUG::2014-05-13 12:43:40,518::mount::226::Storage.Misc.excCmd::(_runcmd) '/usr/bin/sudo -n /bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6,nfsvers=3 dhcp163-66.scl.lab.tlv.redhat.com:/var/lib/exports/iso-20140512085215 /rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215' (cwd None) >Thread-66::DEBUG::2014-05-13 12:43:40,605::hsm::2328::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215 >Thread-66::DEBUG::2014-05-13 12:43:40,607::hsm::2352::Storage.HSM::(__prefetchDomains) Found SD uuids: ('208946c2-5db1-4ef4-80b1-9aeb37f2905e',) >Thread-66::DEBUG::2014-05-13 12:43:40,608::hsm::2408::Storage.HSM::(connectStorageServer) knownSDs: {1ee9ca05-ac9d-4d53-ac4b-6afbe1da0850: storage.nfsSD.findDomain, 1bc6ed6b-8167-4de8-bc1e-ed625a0274d3: storage.blockSD.findDomain, 49db7ad8-2283-4caf-ad27-5dac5b9cb950: storage.blockSD.findDomain, 2874d6f5-ac53-466a-973b-04ddc3beaa24: storage.blockSD.findDomain, 3e8d0026-71e2-43b8-a6cd-853b8861201b: storage.nfsSD.findDomain, 689a6ef7-211a-4bcb-8ac1-42f5cc467967: storage.nfsSD.findDomain, d5af43fc-0c38-4d75-89ad-216daa092503: storage.nfsSD.findDomain, 598916de-8a64-4485-bccb-bc3cbb623569: storage.nfsSD.findDomain, f853f6fe-5b39-4f03-b8a5-dad5823f0bb7: storage.blockSD.findDomain, 208946c2-5db1-4ef4-80b1-9aeb37f2905e: storage.nfsSD.findDomain, bfd13673-a430-442d-beb5-dd42d2107b7d: storage.nfsSD.findDomain, 572b0496-5eab-41cd-a976-4514f111898e: storage.nfsSD.findDomain, 23e7e31a-283a-4398-920a-b3ba8038cb9f: storage.blockSD.findDomain, e4894e28-76eb-4855-bbd1-580c11d4ad51: storage.nfsSD.findDomain, 41830dad-f012-4b4d-b636-863f66d5ca4b: storage.blockSD.findDomain, 6be03a71-472c-4714-bf93-340fdf604955: storage.blockSD.findDomain, c7b37830-fa22-4e63-a8e6-ed70e742a188: storage.nfsSD.findDomain, 9f4c0902-8dba-44c3-8181-9a6b649c61ff: storage.nfsSD.findDomain} >Thread-66::INFO::2014-05-13 12:43:40,608::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '036b03a4-bbab-4c8a-b584-2557b381995a'}]} >Thread-66::DEBUG::2014-05-13 12:43:40,608::task::1185::TaskManager.Task::(prepare) Task=`97fe9167-8a2b-4dc7-bbb3-e30c212536ba`::finished: {'statuslist': [{'status': 0, 'id': '036b03a4-bbab-4c8a-b584-2557b381995a'}]} >Thread-66::DEBUG::2014-05-13 12:43:40,608::task::595::TaskManager.Task::(_updateState) Task=`97fe9167-8a2b-4dc7-bbb3-e30c212536ba`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:43:40,609::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:43:40,609::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:43:40,609::task::990::TaskManager.Task::(_decref) Task=`97fe9167-8a2b-4dc7-bbb3-e30c212536ba`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:43:40,659::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:40,739::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0369072 s, 27.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:43:41,723::task::595::TaskManager.Task::(_updateState) Task=`9556c85a-0ef2-447b-86ee-2e8fa955f01b`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:43:41,724::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:43:41,724::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000723408', 'lastCheck': '7.0', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000354644', 'lastCheck': '10.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:43:41,724::task::1185::TaskManager.Task::(prepare) Task=`9556c85a-0ef2-447b-86ee-2e8fa955f01b`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000723408', 'lastCheck': '7.0', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000354644', 'lastCheck': '10.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:43:41,724::task::595::TaskManager.Task::(_updateState) Task=`9556c85a-0ef2-447b-86ee-2e8fa955f01b`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:43:41,725::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:43:41,725::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:43:41,725::task::990::TaskManager.Task::(_decref) Task=`9556c85a-0ef2-447b-86ee-2e8fa955f01b`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:43:41,782::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:43:41,791::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000413715 s, 841 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:43:42,752::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:42,831::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319728 s, 32.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:43:42,954::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:43:44,765::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:43:44,774::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000722651 s, 5.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:43:44,847::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:44,939::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0493888 s, 20.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:43:45,982::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:43:46,952::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:47,035::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0386174 s, 26.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:43:49,002::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:43:49,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:49,125::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323956 s, 31.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:43:51,139::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:51,234::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0518959 s, 19.7 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:43:51,803::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:43:51,811::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.00040677 s, 856 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:43:52,022::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:43:53,248::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:53,327::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0366348 s, 28.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:43:54,783::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:43:54,791::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000770842 s, 5.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:43:55,042::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:43:55,340::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:55,422::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0378354 s, 27.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:43:56,870::task::595::TaskManager.Task::(_updateState) Task=`a3909605-a7af-46f0-a93b-1c365b1f3158`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:43:56,870::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:43:56,870::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000770842', 'lastCheck': '2.1', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00040677', 'lastCheck': '5.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:43:56,870::task::1185::TaskManager.Task::(prepare) Task=`a3909605-a7af-46f0-a93b-1c365b1f3158`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000770842', 'lastCheck': '2.1', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00040677', 'lastCheck': '5.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:43:56,871::task::595::TaskManager.Task::(_updateState) Task=`a3909605-a7af-46f0-a93b-1c365b1f3158`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:43:56,871::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:43:56,871::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:43:56,871::task::990::TaskManager.Task::(_decref) Task=`a3909605-a7af-46f0-a93b-1c365b1f3158`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:43:57,435::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:57,513::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383661 s, 26.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:43:58,059::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:43:59,521::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:43:59,595::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314436 s, 32.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:44:01,078::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:44:01,608::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:01,682::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309417 s, 33.1 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:44:01,828::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:44:01,836::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000475345 s, 732 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:44:03,695::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:03,783::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0444026 s, 23.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:44:04,095::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:44:04,800::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:44:04,808::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000731608 s, 5.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:44:05,796::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:05,878::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0386332 s, 26.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:44:07,125::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:44:07,891::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:07,970::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379434 s, 27.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:44:09,983::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:10,062::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.03334 s, 30.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:44:10,145::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-344::DEBUG::2014-05-13 12:44:11,848::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:44:11,857::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000409794 s, 849 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:44:12,013::task::595::TaskManager.Task::(_updateState) Task=`dcbc49b7-afdd-4eb5-8c20-a0ecf1b5b2c9`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:44:12,013::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:44:12,014::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000731608', 'lastCheck': '7.2', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000409794', 'lastCheck': '0.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:44:12,014::task::1185::TaskManager.Task::(prepare) Task=`dcbc49b7-afdd-4eb5-8c20-a0ecf1b5b2c9`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000731608', 'lastCheck': '7.2', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000409794', 'lastCheck': '0.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:44:12,014::task::595::TaskManager.Task::(_updateState) Task=`dcbc49b7-afdd-4eb5-8c20-a0ecf1b5b2c9`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:44:12,014::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:44:12,014::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:44:12,015::task::990::TaskManager.Task::(_decref) Task=`dcbc49b7-afdd-4eb5-8c20-a0ecf1b5b2c9`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:44:12,075::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:12,156::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037036 s, 27.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:44:13,164::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:44:14,170::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:14,248::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0326951 s, 31.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:44:14,816::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:44:14,824::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000686031 s, 6.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:44:16,185::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:44:16,261::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:16,336::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0333335 s, 30.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:44:18,349::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:18,443::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0507643 s, 20.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:44:19,546::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:44:20,456::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:20,529::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.030925 s, 33.1 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:44:21,869::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:44:21,878::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.00056115 s, 620 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:44:22,542::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:44:22,564::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:44:22,625::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0394737 s, 25.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:44:24,638::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:24,714::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0325593 s, 31.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:44:24,832::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:44:24,840::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000644254 s, 6.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:44:25,581::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:44:26,727::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:26,810::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373704 s, 27.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:44:27,159::task::595::TaskManager.Task::(_updateState) Task=`f6cd1203-d06f-4ee9-bf81-a50a42cb4ab3`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:44:27,160::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:44:27,160::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000644254', 'lastCheck': '2.3', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00056115', 'lastCheck': '5.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:44:27,160::task::1185::TaskManager.Task::(prepare) Task=`f6cd1203-d06f-4ee9-bf81-a50a42cb4ab3`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000644254', 'lastCheck': '2.3', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00056115', 'lastCheck': '5.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:44:27,160::task::595::TaskManager.Task::(_updateState) Task=`f6cd1203-d06f-4ee9-bf81-a50a42cb4ab3`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:44:27,160::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:44:27,161::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:44:27,161::task::990::TaskManager.Task::(_decref) Task=`f6cd1203-d06f-4ee9-bf81-a50a42cb4ab3`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:44:28,599::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:44:28,828::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:28,902::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314892 s, 32.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:44:30,915::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:30,993::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.032833 s, 31.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:44:31,617::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-344::DEBUG::2014-05-13 12:44:31,889::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:44:31,897::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000514764 s, 676 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:44:33,017::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:33,380::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.319205 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:44:34,635::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:44:34,848::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:44:34,855::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000563651 s, 7.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:44:35,393::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:35,474::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384154 s, 26.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:44:37,487::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:37,566::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364076 s, 28.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:44:37,653::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:44:39,580::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:39,656::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320885 s, 31.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:44:40,671::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:44:41,669::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:41,748::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036183 s, 28.3 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:44:41,910::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:44:41,919::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000537979 s, 647 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:44:42,297::task::595::TaskManager.Task::(_updateState) Task=`8291425c-4530-4c1d-85a7-7fd6522e9181`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:44:42,297::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:44:42,297::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000563651', 'lastCheck': '7.4', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000537979', 'lastCheck': '0.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:44:42,297::task::1185::TaskManager.Task::(prepare) Task=`8291425c-4530-4c1d-85a7-7fd6522e9181`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000563651', 'lastCheck': '7.4', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000537979', 'lastCheck': '0.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:44:42,298::task::595::TaskManager.Task::(_updateState) Task=`8291425c-4530-4c1d-85a7-7fd6522e9181`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:44:42,298::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:44:42,298::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:44:42,298::task::990::TaskManager.Task::(_decref) Task=`8291425c-4530-4c1d-85a7-7fd6522e9181`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:44:43,689::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:44:43,761::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:43,837::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0324396 s, 31.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:44:44,864::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:44:44,872::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000719984 s, 5.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:44:45,850::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:46,205::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.311345 s, 3.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:44:46,708::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:44:48,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:48,298::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.038792 s, 26.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:44:49,726::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:44:50,311::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:50,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0335842 s, 30.5 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:44:51,934::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:44:51,943::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000318305 s, 1.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:44:52,400::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:52,481::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382493 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:44:52,744::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:44:54,494::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:54,572::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0392381 s, 26.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:44:54,880::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:44:54,888::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000616596 s, 6.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:44:55,768::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:44:56,581::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:56,661::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382417 s, 26.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:44:57,459::task::595::TaskManager.Task::(_updateState) Task=`74ca5fda-bfed-467d-a343-f3ed31c9cdf0`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:44:57,459::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:44:57,459::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000616596', 'lastCheck': '2.6', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000318305', 'lastCheck': '5.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:44:57,460::task::1185::TaskManager.Task::(prepare) Task=`74ca5fda-bfed-467d-a343-f3ed31c9cdf0`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000616596', 'lastCheck': '2.6', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000318305', 'lastCheck': '5.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:44:57,460::task::595::TaskManager.Task::(_updateState) Task=`74ca5fda-bfed-467d-a343-f3ed31c9cdf0`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:44:57,460::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:44:57,460::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:44:57,460::task::990::TaskManager.Task::(_decref) Task=`74ca5fda-bfed-467d-a343-f3ed31c9cdf0`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:44:58,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:44:58,762::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0463078 s, 22.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:44:58,788::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:45:00,775::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:00,852::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0325136 s, 31.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:45:01,807::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-344::DEBUG::2014-05-13 12:45:01,992::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:45:02,001::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000510833 s, 681 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:45:02,865::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:02,948::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.039014 s, 26.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:45:04,835::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:45:04,896::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:45:04,904::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000664547 s, 6.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:45:04,961::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:05,041::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0368101 s, 27.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:45:07,054::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:07,153::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0445169 s, 23.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:45:07,853::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:45:09,166::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:09,246::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0385922 s, 26.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:45:10,871::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:45:11,259::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:11,335::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0339991 s, 30.1 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:45:12,018::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:45:12,026::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000449101 s, 775 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:45:12,594::task::595::TaskManager.Task::(_updateState) Task=`361361b8-5c80-4297-9fc2-524fdd6b95b7`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:45:12,594::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:45:12,594::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000664547', 'lastCheck': '7.7', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000449101', 'lastCheck': '0.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:45:12,594::task::1185::TaskManager.Task::(prepare) Task=`361361b8-5c80-4297-9fc2-524fdd6b95b7`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000664547', 'lastCheck': '7.7', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000449101', 'lastCheck': '0.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:45:12,595::task::595::TaskManager.Task::(_updateState) Task=`361361b8-5c80-4297-9fc2-524fdd6b95b7`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:45:12,595::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:45:12,595::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:45:12,595::task::990::TaskManager.Task::(_decref) Task=`361361b8-5c80-4297-9fc2-524fdd6b95b7`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:45:13,348::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:13,436::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0441555 s, 23.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:45:13,888::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:45:14,909::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 41830dad-f012-4b4d-b636-863f66d5ca4b >Thread-322::DEBUG::2014-05-13 12:45:14,909::misc::762::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-322::DEBUG::2014-05-13 12:45:14,909::misc::764::SamplingMethod::(__call__) Got in to sampling method >Thread-322::DEBUG::2014-05-13 12:45:14,909::misc::762::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-322::DEBUG::2014-05-13 12:45:14,910::misc::764::SamplingMethod::(__call__) Got in to sampling method >Thread-322::DEBUG::2014-05-13 12:45:14,910::iscsi::407::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-322::DEBUG::2014-05-13 12:45:14,910::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-322::DEBUG::2014-05-13 12:45:14,934::misc::772::SamplingMethod::(__call__) Returning last result >Thread-322::DEBUG::2014-05-13 12:45:14,934::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >Thread-322::DEBUG::2014-05-13 12:45:15,236::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:45:15,236::lvm::497::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:45:15,237::lvm::499::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:45:15,237::lvm::508::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:45:15,238::lvm::510::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:45:15,238::lvm::528::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:45:15,238::lvm::530::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:45:15,239::misc::772::SamplingMethod::(__call__) Returning last result >Thread-322::DEBUG::2014-05-13 12:45:15,239::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:45:15,242::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:15,449::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 12:45:15,532::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:45:15,536::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:45:15,537::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-322::DEBUG::2014-05-13 12:45:15,558::blockSD::336::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:45:15,573::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000515774 s, 4.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:45:15,575::misc::287::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000515774 s, 4.0 MB/s'], size: 2048 >Thread-322::DEBUG::2014-05-13 12:45:15,576::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-322::WARNING::2014-05-13 12:45:15,577::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-322::DEBUG::2014-05-13 12:45:15,578::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-322::DEBUG::2014-05-13 12:45:15,581::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:45:15,583::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:45:15,584::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:45:15,584::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:45:15,585::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:45:15,586::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:15,594::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0390179 s, 26.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:45:15,843::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:45:15,844::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:45:15,845::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'LOCKPOLICY=', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', 'DESCRIPTION=ISCSI', 'LOGBLKSIZE=512', 'TYPE=ISCSI', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'PHYBLKSIZE=512', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', 'CLASS=Data', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'POOL_DESCRIPTION=DC_2', 'MASTER_VERSION=1', 'ROLE=Master', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', '_SHA_CKSUM=755a5160f5b8caaab19e905b201f8ce3ebcf5fb0', u'POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active'] >Thread-322::INFO::2014-05-13 12:45:15,846::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_imageNS already registered >Thread-322::INFO::2014-05-13 12:45:15,846::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_volumeNS already registered >Thread-322::INFO::2014-05-13 12:45:15,846::blockSD::452::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_lvmActivationNS already registered >Thread-322::DEBUG::2014-05-13 12:45:15,847::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 12:45:16,053::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:45:16,057::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:45:16,065::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000660209 s, 6.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:45:16,907::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:45:17,603::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:17,684::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373585 s, 27.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:45:19,697::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:19,772::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0330922 s, 30.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:45:19,930::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:45:21,785::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:21,862::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0338343 s, 30.3 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:45:22,039::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:45:22,048::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000458736 s, 759 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:45:22,950::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:45:23,875::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:23,957::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0389908 s, 26.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:45:25,968::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:45:25,969::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:26,068::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0498356 s, 20.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:45:26,097::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:45:26,114::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00058067 s, 7.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:45:27,729::task::595::TaskManager.Task::(_updateState) Task=`9e493179-df84-4415-9561-771edfac88ca`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:45:27,729::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:45:27,730::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00058067', 'lastCheck': '1.6', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000458736', 'lastCheck': '5.7', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:45:27,730::task::1185::TaskManager.Task::(prepare) Task=`9e493179-df84-4415-9561-771edfac88ca`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00058067', 'lastCheck': '1.6', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000458736', 'lastCheck': '5.7', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:45:27,730::task::595::TaskManager.Task::(_updateState) Task=`9e493179-df84-4415-9561-771edfac88ca`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:45:27,730::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:45:27,730::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:45:27,730::task::990::TaskManager.Task::(_decref) Task=`9e493179-df84-4415-9561-771edfac88ca`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:45:28,144::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:28,220::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319422 s, 32.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:45:28,991::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:45:30,233::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:30,315::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374183 s, 27.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:45:32,014::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-344::DEBUG::2014-05-13 12:45:32,058::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:45:32,066::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000484762 s, 718 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:45:32,328::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:32,407::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361586 s, 28.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:45:34,420::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:34,496::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0336512 s, 30.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:45:35,033::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:45:36,139::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:45:36,147::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000684729 s, 6.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:45:36,509::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:36,590::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.03856 s, 26.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:45:38,052::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:45:38,604::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:38,685::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0392993 s, 26.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:45:40,698::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:40,788::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0457083 s, 22.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:45:41,070::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-344::DEBUG::2014-05-13 12:45:42,077::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:45:42,086::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000419266 s, 830 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:45:42,801::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 12:45:42,868::task::595::TaskManager.Task::(_updateState) Task=`3785963c-ae8c-4c8a-81f2-b757931c54fa`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:45:42,869::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:45:42,871::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000684729', 'lastCheck': '6.7', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419266', 'lastCheck': '0.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:45:42,871::task::1185::TaskManager.Task::(prepare) Task=`3785963c-ae8c-4c8a-81f2-b757931c54fa`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000684729', 'lastCheck': '6.7', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419266', 'lastCheck': '0.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:45:42,872::task::595::TaskManager.Task::(_updateState) Task=`3785963c-ae8c-4c8a-81f2-b757931c54fa`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:45:42,873::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:45:42,874::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:45:42,875::task::990::TaskManager.Task::(_decref) Task=`3785963c-ae8c-4c8a-81f2-b757931c54fa`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:45:42,905::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323642 s, 31.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:45:44,091::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:45:44,922::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:45,005::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0405039 s, 25.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:45:46,155::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:45:46,163::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000653743 s, 6.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:45:47,018::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:47,097::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373542 s, 27.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:45:47,113::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:45:49,110::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:49,188::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0348856 s, 29.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:45:50,132::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:45:51,201::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:51,293::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037662 s, 27.2 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:45:52,096::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:45:52,104::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000444835 s, 782 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:45:53,150::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:45:53,306::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:53,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379823 s, 27.0 MB/s\n'; <rc> = 0 >Thread-338::ERROR::2014-05-13 12:45:53,664::task::866::TaskManager.Task::(_setError) Task=`438ca0f8-a0ad-4b41-9ff0-4e06d7e3432a`::Unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/storage/task.py", line 873, in _run > return fn(*args, **kargs) > File "/usr/share/vdsm/logUtils.py", line 45, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/storage/hsm.py", line 1200, in attachStorageDomain > pool.attachSD(sdUUID) > File "/usr/share/vdsm/storage/securable.py", line 75, in wrapper > return method(self, *args, **kwargs) > File "/usr/share/vdsm/storage/sp.py", line 890, in attachSD > dom.acquireHostId(self.id) > File "/usr/share/vdsm/storage/sd.py", line 468, in acquireHostId > self._clusterLock.acquireHostId(hostId, async) > File "/usr/share/vdsm/storage/clusterlock.py", line 199, in acquireHostId > raise se.AcquireHostIdFailure(self._sdUUID, e) >AcquireHostIdFailure: Cannot acquire host id: ('689a6ef7-211a-4bcb-8ac1-42f5cc467967', SanlockException(-262, 'Sanlock lockspace add failure', 'Sanlock exception')) >Thread-338::DEBUG::2014-05-13 12:45:53,664::task::885::TaskManager.Task::(_run) Task=`438ca0f8-a0ad-4b41-9ff0-4e06d7e3432a`::Task._run: 438ca0f8-a0ad-4b41-9ff0-4e06d7e3432a ('689a6ef7-211a-4bcb-8ac1-42f5cc467967', 'aa069361-8e06-4459-a17b-748c0a2a828e') {} failed - stopping task >Thread-338::DEBUG::2014-05-13 12:45:53,664::task::1211::TaskManager.Task::(stop) Task=`438ca0f8-a0ad-4b41-9ff0-4e06d7e3432a`::stopping in state preparing (force False) >Thread-338::DEBUG::2014-05-13 12:45:53,665::task::990::TaskManager.Task::(_decref) Task=`438ca0f8-a0ad-4b41-9ff0-4e06d7e3432a`::ref 1 aborting True >Thread-338::INFO::2014-05-13 12:45:53,665::task::1168::TaskManager.Task::(prepare) Task=`438ca0f8-a0ad-4b41-9ff0-4e06d7e3432a`::aborting: Task is aborted: 'Cannot acquire host id' - code 661 >Thread-338::DEBUG::2014-05-13 12:45:53,665::task::1173::TaskManager.Task::(prepare) Task=`438ca0f8-a0ad-4b41-9ff0-4e06d7e3432a`::Prepare: aborted: Cannot acquire host id >Thread-338::DEBUG::2014-05-13 12:45:53,665::task::990::TaskManager.Task::(_decref) Task=`438ca0f8-a0ad-4b41-9ff0-4e06d7e3432a`::ref 0 aborting True >Thread-338::DEBUG::2014-05-13 12:45:53,665::task::925::TaskManager.Task::(_doAbort) Task=`438ca0f8-a0ad-4b41-9ff0-4e06d7e3432a`::Task._doAbort: force False >Thread-338::DEBUG::2014-05-13 12:45:53,665::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:45:53,666::task::595::TaskManager.Task::(_updateState) Task=`438ca0f8-a0ad-4b41-9ff0-4e06d7e3432a`::moving from state preparing -> state aborting >Thread-338::DEBUG::2014-05-13 12:45:53,666::task::550::TaskManager.Task::(__state_aborting) Task=`438ca0f8-a0ad-4b41-9ff0-4e06d7e3432a`::_aborting: recover policy none >Thread-338::DEBUG::2014-05-13 12:45:53,666::task::595::TaskManager.Task::(_updateState) Task=`438ca0f8-a0ad-4b41-9ff0-4e06d7e3432a`::moving from state aborting -> state failed >Thread-338::DEBUG::2014-05-13 12:45:53,666::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.689a6ef7-211a-4bcb-8ac1-42f5cc467967': < ResourceRef 'Storage.689a6ef7-211a-4bcb-8ac1-42f5cc467967', isValid: 'True' obj: 'None'>, 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:45:53,666::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:45:53,667::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.689a6ef7-211a-4bcb-8ac1-42f5cc467967' >Thread-338::DEBUG::2014-05-13 12:45:53,667::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.689a6ef7-211a-4bcb-8ac1-42f5cc467967' (0 active users) >Thread-338::DEBUG::2014-05-13 12:45:53,667::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.689a6ef7-211a-4bcb-8ac1-42f5cc467967' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:45:53,667::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.689a6ef7-211a-4bcb-8ac1-42f5cc467967', Clearing records. >Thread-338::DEBUG::2014-05-13 12:45:53,668::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:45:53,668::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:45:53,668::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:45:53,668::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::ERROR::2014-05-13 12:45:53,668::dispatcher::65::Storage.Dispatcher.Protect::(run) {'status': {'message': "Cannot acquire host id: ('689a6ef7-211a-4bcb-8ac1-42f5cc467967', SanlockException(-262, 'Sanlock lockspace add failure', 'Sanlock exception'))", 'code': 661}} >Thread-66::DEBUG::2014-05-13 12:45:53,716::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:45:53,716::task::595::TaskManager.Task::(_updateState) Task=`c49cbe00-375c-4200-b039-323a83055a7d`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:45:53,716::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:45:53,717::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:45:53,717::task::1185::TaskManager.Task::(prepare) Task=`c49cbe00-375c-4200-b039-323a83055a7d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:45:53,717::task::595::TaskManager.Task::(_updateState) Task=`c49cbe00-375c-4200-b039-323a83055a7d`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:45:53,717::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:45:53,717::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:45:53,717::task::990::TaskManager.Task::(_decref) Task=`c49cbe00-375c-4200-b039-323a83055a7d`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:45:53,722::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:45:53,722::task::595::TaskManager.Task::(_updateState) Task=`b6eb1f4a-d872-4da3-b6a3-d3603678e7ec`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:45:53,722::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:45:53,723::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`7e2dd7cd-d074-4bac-90ca-b420da801c98`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:45:53,723::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:45:53,723::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:45:53,723::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`7e2dd7cd-d074-4bac-90ca-b420da801c98`::Granted request >Thread-338::DEBUG::2014-05-13 12:45:53,724::task::827::TaskManager.Task::(resourceAcquired) Task=`b6eb1f4a-d872-4da3-b6a3-d3603678e7ec`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:45:53,724::task::990::TaskManager.Task::(_decref) Task=`b6eb1f4a-d872-4da3-b6a3-d3603678e7ec`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:45:53,725::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': u'41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:45:53,725::task::1185::TaskManager.Task::(prepare) Task=`b6eb1f4a-d872-4da3-b6a3-d3603678e7ec`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': u'41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:45:53,725::task::595::TaskManager.Task::(_updateState) Task=`b6eb1f4a-d872-4da3-b6a3-d3603678e7ec`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:45:53,725::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:45:53,726::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:45:53,726::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:45:53,726::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:45:53,726::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:45:53,726::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:45:53,727::task::990::TaskManager.Task::(_decref) Task=`b6eb1f4a-d872-4da3-b6a3-d3603678e7ec`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:45:53,761::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3d8543a1] >Thread-338::DEBUG::2014-05-13 12:45:53,762::task::595::TaskManager.Task::(_updateState) Task=`43580623-6cf0-4fcd-97e5-4c67d2d6ffed`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:45:53,762::logUtils::44::dispatcher::(wrapper) Run and protect: attachStorageDomain(sdUUID='208946c2-5db1-4ef4-80b1-9aeb37f2905e', spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:45:53,762::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0acd1e04-bdfa-4135-ae44-441457924fa7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1197' at 'attachStorageDomain' >Thread-338::DEBUG::2014-05-13 12:45:53,762::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'exclusive' >Thread-338::DEBUG::2014-05-13 12:45:53,763::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'exclusive' (1 active user) >Thread-338::DEBUG::2014-05-13 12:45:53,763::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0acd1e04-bdfa-4135-ae44-441457924fa7`::Granted request >Thread-338::DEBUG::2014-05-13 12:45:53,763::task::827::TaskManager.Task::(resourceAcquired) Task=`43580623-6cf0-4fcd-97e5-4c67d2d6ffed`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (exclusive) >Thread-338::DEBUG::2014-05-13 12:45:53,763::task::990::TaskManager.Task::(_decref) Task=`43580623-6cf0-4fcd-97e5-4c67d2d6ffed`::ref 1 aborting False >Thread-338::DEBUG::2014-05-13 12:45:53,764::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e`ReqID=`d2dcb815-38a7-487a-a3bf-0a5a35a37d90`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1198' at 'attachStorageDomain' >Thread-338::DEBUG::2014-05-13 12:45:53,764::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e' for lock type 'exclusive' >Thread-338::DEBUG::2014-05-13 12:45:53,764::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e' is free. Now locking as 'exclusive' (1 active user) >Thread-338::DEBUG::2014-05-13 12:45:53,764::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e`ReqID=`d2dcb815-38a7-487a-a3bf-0a5a35a37d90`::Granted request >Thread-338::DEBUG::2014-05-13 12:45:53,764::task::827::TaskManager.Task::(resourceAcquired) Task=`43580623-6cf0-4fcd-97e5-4c67d2d6ffed`::_resourcesAcquired: Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e (exclusive) >Thread-338::DEBUG::2014-05-13 12:45:53,765::task::990::TaskManager.Task::(_decref) Task=`43580623-6cf0-4fcd-97e5-4c67d2d6ffed`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:45:53,765::sp::869::Storage.StoragePool::(attachSD) sdUUID=208946c2-5db1-4ef4-80b1-9aeb37f2905e spUUID=aa069361-8e06-4459-a17b-748c0a2a828e >Thread-338::DEBUG::2014-05-13 12:45:53,909::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e >Thread-338::DEBUG::2014-05-13 12:45:54,048::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-338::DEBUG::2014-05-13 12:45:54,079::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=ISO_DOMAIN', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=', 'REMOTE_PATH=dhcp163-66.scl.lab.tlv.redhat.com:/var/lib/exports/iso-20140512085215', 'ROLE=Regular', 'SDUUID=208946c2-5db1-4ef4-80b1-9aeb37f2905e', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=3b9a2e9e7582d866e4efdb542523b2cadca4f980'] >Thread-338::DEBUG::2014-05-13 12:45:54,081::fileSD::609::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-338::DEBUG::2014-05-13 12:45:54,082::resourceManager::421::ResourceManager::(registerNamespace) Registering namespace '208946c2-5db1-4ef4-80b1-9aeb37f2905e_imageNS' >Thread-338::DEBUG::2014-05-13 12:45:54,082::resourceManager::421::ResourceManager::(registerNamespace) Registering namespace '208946c2-5db1-4ef4-80b1-9aeb37f2905e_volumeNS' >Thread-338::DEBUG::2014-05-13 12:45:54,083::clusterlock::86::SafeLease::(acquireHostId) Host id for domain 208946c2-5db1-4ef4-80b1-9aeb37f2905e successfully acquired (id: 1) >Thread-338::DEBUG::2014-05-13 12:45:54,083::clusterlock::100::SafeLease::(acquire) Acquiring cluster lock for domain 208946c2-5db1-4ef4-80b1-9aeb37f2905e >Thread-338::DEBUG::2014-05-13 12:45:54,084::clusterlock::114::Storage.Misc.excCmd::(acquire) '/usr/bin/sudo -n /usr/bin/setsid /usr/bin/ionice -c 1 -n 0 /bin/su vdsm -s /bin/sh -c "/usr/libexec/vdsm/spmprotect.sh start 208946c2-5db1-4ef4-80b1-9aeb37f2905e 1 5 /rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/leases 5000 1000 3"' (cwd /usr/libexec/vdsm) >Dummy-337::DEBUG::2014-05-13 12:45:55,400::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:55,487::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0460623 s, 22.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:45:56,166::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:45:56,374::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:45:56,382::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000607312 s, 6.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:45:57,500::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:57,582::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0370326 s, 27.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:45:58,026::task::595::TaskManager.Task::(_updateState) Task=`6330cbc2-4672-41c3-a69b-dd99c3823e3d`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:45:58,026::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:45:58,026::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000607312', 'lastCheck': '1.6', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000444835', 'lastCheck': '5.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:45:58,026::task::1185::TaskManager.Task::(prepare) Task=`6330cbc2-4672-41c3-a69b-dd99c3823e3d`::finished: {'41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000607312', 'lastCheck': '1.6', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000444835', 'lastCheck': '5.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:45:58,026::task::595::TaskManager.Task::(_updateState) Task=`6330cbc2-4672-41c3-a69b-dd99c3823e3d`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:45:58,027::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:45:58,027::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:45:58,027::task::990::TaskManager.Task::(_decref) Task=`6330cbc2-4672-41c3-a69b-dd99c3823e3d`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:45:59,184::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:45:59,595::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:45:59,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319826 s, 32.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:46:01,686::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:01,760::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311866 s, 32.8 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:46:02,115::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:46:02,124::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.00046253 s, 752 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:46:02,202::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:46:03,773::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:03,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0325308 s, 31.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:46:05,219::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:46:05,864::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:05,944::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361212 s, 28.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:46:06,440::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:46:06,448::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000713713 s, 5.7 MB/s\n'; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:46:07,267::clusterlock::114::Storage.Misc.excCmd::(acquire) SUCCESS: <err> = ''; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:46:07,268::clusterlock::117::SafeLease::(acquire) Clustered lock acquired successfully >Thread-338::DEBUG::2014-05-13 12:46:07,276::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=ISO_DOMAIN', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=', 'REMOTE_PATH=dhcp163-66.scl.lab.tlv.redhat.com:/var/lib/exports/iso-20140512085215', 'ROLE=Regular', 'SDUUID=208946c2-5db1-4ef4-80b1-9aeb37f2905e', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=3b9a2e9e7582d866e4efdb542523b2cadca4f980'] >Thread-338::DEBUG::2014-05-13 12:46:07,277::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-338::DEBUG::2014-05-13 12:46:07,277::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >Thread-338::DEBUG::2014-05-13 12:46:07,277::persistentDict::299::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=ISO_DOMAIN', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', 'REMOTE_PATH=dhcp163-66.scl.lab.tlv.redhat.com:/var/lib/exports/iso-20140512085215', 'ROLE=Regular', 'SDUUID=208946c2-5db1-4ef4-80b1-9aeb37f2905e', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=0be9ca2275e0b4f627ac4aeccb1d1475df0d42f5'] >Thread-338::DEBUG::2014-05-13 12:46:07,285::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-338::DEBUG::2014-05-13 12:46:07,286::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-338::DEBUG::2014-05-13 12:46:07,286::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >Thread-338::DEBUG::2014-05-13 12:46:07,286::persistentDict::299::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=['CLASS=Data', 'DESCRIPTION=ISCSI', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'MASTER_VERSION=1', 'PHYBLKSIZE=512', 'POOL_DESCRIPTION=DC_2', u'POOL_DOMAINS=208946c2-5db1-4ef4-80b1-9aeb37f2905e:Attached,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', 'ROLE=Master', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'TYPE=ISCSI', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', '_SHA_CKSUM=67748bda78b49542e23a3299753611a1070a6aa1'] >Thread-338::DEBUG::2014-05-13 12:46:07,287::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:46:07,287::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:46:07,287::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:46:07,288::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:46:07,288::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:46:07,289::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-338::DEBUG::2014-05-13 12:46:07,504::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:46:07,505::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:46:07,511::blockSD::315::storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT_POOL_DOMAINS=208946c2-5db1-4ef4-80b1-9aeb37f2905e:Attached&44&41830dad-f012-4b4d-b636-863f66d5ca4b:Active&44&572b0496-5eab-41cd-a976-4514f111898e:Active, MDT__SHA_CKSUM=67748bda78b49542e23a3299753611a1070a6aa1 removing=MDT_POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active&44&572b0496-5eab-41cd-a976-4514f111898e:Active, MDT__SHA_CKSUM=755a5160f5b8caaab19e905b201f8ce3ebcf5fb0 >Thread-338::DEBUG::2014-05-13 12:46:07,512::lvm::296::Storage.Misc.excCmd::(cmd) u'/usr/bin/sudo -n /sbin/lvm vgchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --deltag MDT__SHA_CKSUM=755a5160f5b8caaab19e905b201f8ce3ebcf5fb0 --deltag MDT_POOL_DOMAINS=41830dad-f012-4b4d-b636-863f66d5ca4b:Active&44&572b0496-5eab-41cd-a976-4514f111898e:Active --addtag MDT__SHA_CKSUM=67748bda78b49542e23a3299753611a1070a6aa1 --addtag MDT_POOL_DOMAINS=208946c2-5db1-4ef4-80b1-9aeb37f2905e:Attached&44&41830dad-f012-4b4d-b636-863f66d5ca4b:Active&44&572b0496-5eab-41cd-a976-4514f111898e:Active 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-338::DEBUG::2014-05-13 12:46:07,940::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:46:07,941::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:46:07,941::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:46:07,941::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-338::INFO::2014-05-13 12:46:07,942::sp::1113::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e to /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/208946c2-5db1-4ef4-80b1-9aeb37f2905e >Thread-338::INFO::2014-05-13 12:46:07,943::clusterlock::130::SafeLease::(release) Releasing cluster lock for domain 208946c2-5db1-4ef4-80b1-9aeb37f2905e >Thread-338::DEBUG::2014-05-13 12:46:07,943::clusterlock::132::Storage.Misc.excCmd::(release) '/usr/libexec/vdsm/spmstop.sh 208946c2-5db1-4ef4-80b1-9aeb37f2905e' (cwd /usr/libexec/vdsm) >Dummy-337::DEBUG::2014-05-13 12:46:07,956::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:08,054::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0501424 s, 20.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:46:08,246::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-338::DEBUG::2014-05-13 12:46:09,012::clusterlock::132::Storage.Misc.excCmd::(release) SUCCESS: <err> = ''; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:46:09,012::clusterlock::137::SafeLease::(release) Cluster lock released successfully >Thread-338::DEBUG::2014-05-13 12:46:09,012::clusterlock::90::SafeLease::(releaseHostId) Host id for domain 208946c2-5db1-4ef4-80b1-9aeb37f2905e released successfully (id: 1) >Thread-338::DEBUG::2014-05-13 12:46:09,013::misc::762::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-338::DEBUG::2014-05-13 12:46:09,013::misc::764::SamplingMethod::(__call__) Got in to sampling method >Thread-338::DEBUG::2014-05-13 12:46:09,013::misc::772::SamplingMethod::(__call__) Returning last result >Thread-338::INFO::2014-05-13 12:46:09,013::logUtils::47::dispatcher::(wrapper) Run and protect: attachStorageDomain, Return response: None >Thread-338::DEBUG::2014-05-13 12:46:09,014::task::1185::TaskManager.Task::(prepare) Task=`43580623-6cf0-4fcd-97e5-4c67d2d6ffed`::finished: None >Thread-338::DEBUG::2014-05-13 12:46:09,014::task::595::TaskManager.Task::(_updateState) Task=`43580623-6cf0-4fcd-97e5-4c67d2d6ffed`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:46:09,014::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e': < ResourceRef 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e', isValid: 'True' obj: 'None'>, 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:46:09,014::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:46:09,015::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e' >Thread-338::DEBUG::2014-05-13 12:46:09,015::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:46:09,015::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:46:09,015::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:46:09,015::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:46:09,016::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:46:09,016::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:46:09,016::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:46:09,016::task::990::TaskManager.Task::(_decref) Task=`43580623-6cf0-4fcd-97e5-4c67d2d6ffed`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:46:09,073::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:46:09,073::task::595::TaskManager.Task::(_updateState) Task=`8864af00-602f-4762-be4f-3a7dda3561fb`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:46:09,073::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:46:09,074::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:46:09,074::task::1185::TaskManager.Task::(prepare) Task=`8864af00-602f-4762-be4f-3a7dda3561fb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:46:09,074::task::595::TaskManager.Task::(_updateState) Task=`8864af00-602f-4762-be4f-3a7dda3561fb`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:46:09,074::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:46:09,074::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:46:09,074::task::990::TaskManager.Task::(_decref) Task=`8864af00-602f-4762-be4f-3a7dda3561fb`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:46:09,079::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:46:09,079::task::595::TaskManager.Task::(_updateState) Task=`3ab8b49f-6378-4abc-b8ec-f9e0da93c484`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:46:09,079::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:46:09,080::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`699e95a3-437e-49b6-94fd-6c7e26d9381c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:46:09,080::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:46:09,080::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:46:09,081::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`699e95a3-437e-49b6-94fd-6c7e26d9381c`::Granted request >Thread-338::DEBUG::2014-05-13 12:46:09,081::task::827::TaskManager.Task::(resourceAcquired) Task=`3ab8b49f-6378-4abc-b8ec-f9e0da93c484`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:46:09,081::task::990::TaskManager.Task::(_decref) Task=`3ab8b49f-6378-4abc-b8ec-f9e0da93c484`::ref 1 aborting False >Thread-338::DEBUG::2014-05-13 12:46:09,082::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:46:09,082::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-66::DEBUG::2014-05-13 12:46:09,121::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [358e66f] >Thread-66::DEBUG::2014-05-13 12:46:09,122::task::595::TaskManager.Task::(_updateState) Task=`333d933c-ef32-4852-a9f1-1123222ae9fa`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:46:09,122::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': 'dhcp163-66.scl.lab.tlv.redhat.com:/var/lib/exports/iso-20140512085215', 'iqn': '', 'user': '', 'tpgt': '', 'password': '******', 'id': '036b03a4-bbab-4c8a-b584-2557b381995a', 'port': ''}], options=None) >Thread-66::DEBUG::2014-05-13 12:46:09,126::hsm::2328::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215 >Thread-66::DEBUG::2014-05-13 12:46:09,128::hsm::2352::Storage.HSM::(__prefetchDomains) Found SD uuids: ('208946c2-5db1-4ef4-80b1-9aeb37f2905e',) >Thread-66::DEBUG::2014-05-13 12:46:09,128::hsm::2408::Storage.HSM::(connectStorageServer) knownSDs: {1ee9ca05-ac9d-4d53-ac4b-6afbe1da0850: storage.nfsSD.findDomain, 1bc6ed6b-8167-4de8-bc1e-ed625a0274d3: storage.blockSD.findDomain, 49db7ad8-2283-4caf-ad27-5dac5b9cb950: storage.blockSD.findDomain, 2874d6f5-ac53-466a-973b-04ddc3beaa24: storage.blockSD.findDomain, 3e8d0026-71e2-43b8-a6cd-853b8861201b: storage.nfsSD.findDomain, 689a6ef7-211a-4bcb-8ac1-42f5cc467967: storage.nfsSD.findDomain, d5af43fc-0c38-4d75-89ad-216daa092503: storage.nfsSD.findDomain, 598916de-8a64-4485-bccb-bc3cbb623569: storage.nfsSD.findDomain, f853f6fe-5b39-4f03-b8a5-dad5823f0bb7: storage.blockSD.findDomain, 208946c2-5db1-4ef4-80b1-9aeb37f2905e: storage.nfsSD.findDomain, bfd13673-a430-442d-beb5-dd42d2107b7d: storage.nfsSD.findDomain, 572b0496-5eab-41cd-a976-4514f111898e: storage.nfsSD.findDomain, 23e7e31a-283a-4398-920a-b3ba8038cb9f: storage.blockSD.findDomain, e4894e28-76eb-4855-bbd1-580c11d4ad51: storage.nfsSD.findDomain, 41830dad-f012-4b4d-b636-863f66d5ca4b: storage.blockSD.findDomain, 6be03a71-472c-4714-bf93-340fdf604955: storage.blockSD.findDomain, c7b37830-fa22-4e63-a8e6-ed70e742a188: storage.nfsSD.findDomain, 9f4c0902-8dba-44c3-8181-9a6b649c61ff: storage.nfsSD.findDomain} >Thread-66::INFO::2014-05-13 12:46:09,128::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '036b03a4-bbab-4c8a-b584-2557b381995a'}]} >Thread-66::DEBUG::2014-05-13 12:46:09,128::task::1185::TaskManager.Task::(prepare) Task=`333d933c-ef32-4852-a9f1-1123222ae9fa`::finished: {'statuslist': [{'status': 0, 'id': '036b03a4-bbab-4c8a-b584-2557b381995a'}]} >Thread-66::DEBUG::2014-05-13 12:46:09,129::task::595::TaskManager.Task::(_updateState) Task=`333d933c-ef32-4852-a9f1-1123222ae9fa`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:46:09,129::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:46:09,129::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:46:09,129::task::990::TaskManager.Task::(_decref) Task=`333d933c-ef32-4852-a9f1-1123222ae9fa`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:46:09,386::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:46:09,387::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-338::INFO::2014-05-13 12:46:09,388::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Attached,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Attached', 'isoprefix': '', 'alerts': []}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:46:09,389::task::1185::TaskManager.Task::(prepare) Task=`3ab8b49f-6378-4abc-b8ec-f9e0da93c484`::finished: {'info': {'name': 'DC_2', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Attached,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Attached', 'isoprefix': '', 'alerts': []}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:46:09,389::task::595::TaskManager.Task::(_updateState) Task=`3ab8b49f-6378-4abc-b8ec-f9e0da93c484`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:46:09,389::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:46:09,389::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:46:09,390::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:46:09,390::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:46:09,390::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:46:09,390::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:46:09,390::task::990::TaskManager.Task::(_decref) Task=`3ab8b49f-6378-4abc-b8ec-f9e0da93c484`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:46:09,441::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [249764c5] >Thread-338::DEBUG::2014-05-13 12:46:09,441::task::595::TaskManager.Task::(_updateState) Task=`252b48ff-63fd-4da4-8154-d0f62565c533`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:46:09,441::logUtils::44::dispatcher::(wrapper) Run and protect: activateStorageDomain(sdUUID='208946c2-5db1-4ef4-80b1-9aeb37f2905e', spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:46:09,442::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6d04dbc2-0ef9-4e8d-b7f1-e785c9f56db2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1256' at 'activateStorageDomain' >Thread-338::DEBUG::2014-05-13 12:46:09,442::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'exclusive' >Thread-338::DEBUG::2014-05-13 12:46:09,442::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'exclusive' (1 active user) >Thread-338::DEBUG::2014-05-13 12:46:09,442::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6d04dbc2-0ef9-4e8d-b7f1-e785c9f56db2`::Granted request >Thread-338::DEBUG::2014-05-13 12:46:09,442::task::827::TaskManager.Task::(resourceAcquired) Task=`252b48ff-63fd-4da4-8154-d0f62565c533`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (exclusive) >Thread-338::DEBUG::2014-05-13 12:46:09,443::task::990::TaskManager.Task::(_decref) Task=`252b48ff-63fd-4da4-8154-d0f62565c533`::ref 1 aborting False >Thread-338::DEBUG::2014-05-13 12:46:09,443::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e`ReqID=`4ffb571a-330d-4b1b-812f-84916ffc6356`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1257' at 'activateStorageDomain' >Thread-338::DEBUG::2014-05-13 12:46:09,443::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e' for lock type 'exclusive' >Thread-338::DEBUG::2014-05-13 12:46:09,443::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e' is free. Now locking as 'exclusive' (1 active user) >Thread-338::DEBUG::2014-05-13 12:46:09,444::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e`ReqID=`4ffb571a-330d-4b1b-812f-84916ffc6356`::Granted request >Thread-338::DEBUG::2014-05-13 12:46:09,444::task::827::TaskManager.Task::(resourceAcquired) Task=`252b48ff-63fd-4da4-8154-d0f62565c533`::_resourcesAcquired: Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e (exclusive) >Thread-338::DEBUG::2014-05-13 12:46:09,444::task::990::TaskManager.Task::(_decref) Task=`252b48ff-63fd-4da4-8154-d0f62565c533`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:46:09,444::sp::1024::Storage.StoragePool::(activateSD) sdUUID=208946c2-5db1-4ef4-80b1-9aeb37f2905e spUUID=aa069361-8e06-4459-a17b-748c0a2a828e >Thread-338::DEBUG::2014-05-13 12:46:09,444::misc::762::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-338::DEBUG::2014-05-13 12:46:09,445::misc::764::SamplingMethod::(__call__) Got in to sampling method >Thread-338::DEBUG::2014-05-13 12:46:09,445::misc::762::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-338::DEBUG::2014-05-13 12:46:09,445::misc::764::SamplingMethod::(__call__) Got in to sampling method >Thread-338::DEBUG::2014-05-13 12:46:09,445::iscsi::407::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-338::DEBUG::2014-05-13 12:46:09,445::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-338::DEBUG::2014-05-13 12:46:09,468::misc::772::SamplingMethod::(__call__) Returning last result >Thread-338::DEBUG::2014-05-13 12:46:09,468::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >Thread-338::DEBUG::2014-05-13 12:46:09,716::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:46:09,717::lvm::497::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:46:09,717::lvm::499::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:46:09,718::lvm::508::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:46:09,718::lvm::510::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:46:09,719::lvm::528::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:46:09,719::lvm::530::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:46:09,720::misc::772::SamplingMethod::(__call__) Returning last result >Thread-338::DEBUG::2014-05-13 12:46:09,741::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e >Thread-338::DEBUG::2014-05-13 12:46:09,743::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-338::DEBUG::2014-05-13 12:46:09,760::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=ISO_DOMAIN', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', 'REMOTE_PATH=dhcp163-66.scl.lab.tlv.redhat.com:/var/lib/exports/iso-20140512085215', 'ROLE=Regular', 'SDUUID=208946c2-5db1-4ef4-80b1-9aeb37f2905e', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=0be9ca2275e0b4f627ac4aeccb1d1475df0d42f5'] >Thread-338::DEBUG::2014-05-13 12:46:09,763::fileSD::609::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-338::INFO::2014-05-13 12:46:09,763::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 208946c2-5db1-4ef4-80b1-9aeb37f2905e_imageNS already registered >Thread-338::INFO::2014-05-13 12:46:09,764::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 208946c2-5db1-4ef4-80b1-9aeb37f2905e_volumeNS already registered >Thread-338::INFO::2014-05-13 12:46:09,765::sp::1113::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e to /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/208946c2-5db1-4ef4-80b1-9aeb37f2905e >Thread-338::DEBUG::2014-05-13 12:46:09,766::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-338::DEBUG::2014-05-13 12:46:09,767::persistentDict::160::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-338::DEBUG::2014-05-13 12:46:09,767::persistentDict::160::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-338::DEBUG::2014-05-13 12:46:09,767::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-338::DEBUG::2014-05-13 12:46:09,768::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-338::DEBUG::2014-05-13 12:46:09,768::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >Thread-338::DEBUG::2014-05-13 12:46:09,769::persistentDict::299::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=['CLASS=Data', 'DESCRIPTION=ISCSI', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'MASTER_VERSION=1', 'PHYBLKSIZE=512', 'POOL_DESCRIPTION=DC_2', u'POOL_DOMAINS=208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', 'ROLE=Master', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'TYPE=ISCSI', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', '_SHA_CKSUM=985bc0a261dab0d9f33927b045f395c62a74e955'] >Thread-338::DEBUG::2014-05-13 12:46:09,769::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:46:09,770::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:46:09,770::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:46:09,771::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:46:09,771::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:46:09,776::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-338::DEBUG::2014-05-13 12:46:10,029::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:46:10,030::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:46:10,031::blockSD::315::storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT_POOL_DOMAINS=208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active&44&41830dad-f012-4b4d-b636-863f66d5ca4b:Active&44&572b0496-5eab-41cd-a976-4514f111898e:Active, MDT__SHA_CKSUM=985bc0a261dab0d9f33927b045f395c62a74e955 removing=MDT_POOL_DOMAINS=208946c2-5db1-4ef4-80b1-9aeb37f2905e:Attached&44&41830dad-f012-4b4d-b636-863f66d5ca4b:Active&44&572b0496-5eab-41cd-a976-4514f111898e:Active, MDT__SHA_CKSUM=67748bda78b49542e23a3299753611a1070a6aa1 >Thread-338::DEBUG::2014-05-13 12:46:10,032::lvm::296::Storage.Misc.excCmd::(cmd) u'/usr/bin/sudo -n /sbin/lvm vgchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --deltag MDT__SHA_CKSUM=67748bda78b49542e23a3299753611a1070a6aa1 --deltag MDT_POOL_DOMAINS=208946c2-5db1-4ef4-80b1-9aeb37f2905e:Attached&44&41830dad-f012-4b4d-b636-863f66d5ca4b:Active&44&572b0496-5eab-41cd-a976-4514f111898e:Active --addtag MDT_POOL_DOMAINS=208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active&44&41830dad-f012-4b4d-b636-863f66d5ca4b:Active&44&572b0496-5eab-41cd-a976-4514f111898e:Active --addtag MDT__SHA_CKSUM=985bc0a261dab0d9f33927b045f395c62a74e955 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:10,067::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:10,164::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0460877 s, 22.2 MB/s\n'; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:46:10,467::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:46:10,467::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-338::DEBUG::2014-05-13 12:46:10,468::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-338::DEBUG::2014-05-13 12:46:10,468::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-338::DEBUG::2014-05-13 12:46:10,468::misc::762::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-338::DEBUG::2014-05-13 12:46:10,468::misc::764::SamplingMethod::(__call__) Got in to sampling method >Thread-338::INFO::2014-05-13 12:46:10,469::domainMonitor::99::Storage.DomainMonitor::(startMonitoring) Start monitoring 208946c2-5db1-4ef4-80b1-9aeb37f2905e >Thread-357::DEBUG::2014-05-13 12:46:10,469::domainMonitor::165::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for 208946c2-5db1-4ef4-80b1-9aeb37f2905e >Thread-338::DEBUG::2014-05-13 12:46:10,469::sp::1404::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `aa069361-8e06-4459-a17b-748c0a2a828e` started monitoring domain `208946c2-5db1-4ef4-80b1-9aeb37f2905e` >Thread-338::DEBUG::2014-05-13 12:46:10,471::misc::772::SamplingMethod::(__call__) Returning last result >Thread-338::INFO::2014-05-13 12:46:10,472::logUtils::47::dispatcher::(wrapper) Run and protect: activateStorageDomain, Return response: None >Thread-338::DEBUG::2014-05-13 12:46:10,474::task::1185::TaskManager.Task::(prepare) Task=`252b48ff-63fd-4da4-8154-d0f62565c533`::finished: None >Thread-338::DEBUG::2014-05-13 12:46:10,476::task::595::TaskManager.Task::(_updateState) Task=`252b48ff-63fd-4da4-8154-d0f62565c533`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:46:10,478::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e': < ResourceRef 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e', isValid: 'True' obj: 'None'>, 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:46:10,480::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:46:10,483::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e' >Thread-338::DEBUG::2014-05-13 12:46:10,486::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:46:10,487::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:46:10,489::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:46:10,493::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:46:10,494::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:46:10,494::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:46:10,494::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:46:10,495::task::990::TaskManager.Task::(_decref) Task=`252b48ff-63fd-4da4-8154-d0f62565c533`::ref 0 aborting False >Thread-357::DEBUG::2014-05-13 12:46:10,504::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-66::DEBUG::2014-05-13 12:46:10,514::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [249764c5] >Thread-66::DEBUG::2014-05-13 12:46:10,517::task::595::TaskManager.Task::(_updateState) Task=`928ac3dc-8868-4486-9af1-af52b42c8e0a`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:46:10,519::logUtils::44::dispatcher::(wrapper) Run and protect: refreshStoragePool(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', msdUUID='41830dad-f012-4b4d-b636-863f66d5ca4b', masterVersion=1, options=None) >Thread-66::DEBUG::2014-05-13 12:46:10,522::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`95ae4396-cfd6-4459-8313-3841074f6bb4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '870' at 'refreshStoragePool' >Thread-66::DEBUG::2014-05-13 12:46:10,523::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-66::DEBUG::2014-05-13 12:46:10,526::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-357::DEBUG::2014-05-13 12:46:10,526::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000335891 s, 1.2 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:46:10,527::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`95ae4396-cfd6-4459-8313-3841074f6bb4`::Granted request >Thread-66::DEBUG::2014-05-13 12:46:10,529::task::827::TaskManager.Task::(resourceAcquired) Task=`928ac3dc-8868-4486-9af1-af52b42c8e0a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-66::DEBUG::2014-05-13 12:46:10,529::task::990::TaskManager.Task::(_decref) Task=`928ac3dc-8868-4486-9af1-af52b42c8e0a`::ref 1 aborting False >Thread-66::INFO::2014-05-13 12:46:10,529::hsm::879::Storage.HSM::(refreshStoragePool) Ignoring the refreshStoragePool request (the host is the SPM) >Thread-66::INFO::2014-05-13 12:46:10,530::logUtils::47::dispatcher::(wrapper) Run and protect: refreshStoragePool, Return response: None >Thread-357::DEBUG::2014-05-13 12:46:10,532::domainMonitor::247::Storage.DomainMonitorThread::(_monitorDomain) Domain 208946c2-5db1-4ef4-80b1-9aeb37f2905e changed its status to Valid >Thread-66::DEBUG::2014-05-13 12:46:10,532::task::1185::TaskManager.Task::(prepare) Task=`928ac3dc-8868-4486-9af1-af52b42c8e0a`::finished: None >Thread-66::DEBUG::2014-05-13 12:46:10,534::task::595::TaskManager.Task::(_updateState) Task=`928ac3dc-8868-4486-9af1-af52b42c8e0a`::moving from state preparing -> state finished >Thread-358::DEBUG::2014-05-13 12:46:10,534::misc::884::Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Emitting event >Thread-66::DEBUG::2014-05-13 12:46:10,535::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-358::DEBUG::2014-05-13 12:46:10,535::misc::894::Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `_onDomainStateChange` >Thread-66::DEBUG::2014-05-13 12:46:10,536::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-358::DEBUG::2014-05-13 12:46:10,537::misc::894::Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `contEIOVms` >Thread-66::DEBUG::2014-05-13 12:46:10,538::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-359::INFO::2014-05-13 12:46:10,538::sp::1113::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e to /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/208946c2-5db1-4ef4-80b1-9aeb37f2905e >Thread-358::DEBUG::2014-05-13 12:46:10,540::misc::894::Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `_upgradePoolDomain` >Thread-66::DEBUG::2014-05-13 12:46:10,542::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-358::DEBUG::2014-05-13 12:46:10,543::misc::904::Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Event emitted >Thread-66::DEBUG::2014-05-13 12:46:10,544::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-66::DEBUG::2014-05-13 12:46:10,545::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-66::DEBUG::2014-05-13 12:46:10,545::task::990::TaskManager.Task::(_decref) Task=`928ac3dc-8868-4486-9af1-af52b42c8e0a`::ref 0 aborting False >Thread-360::INFO::2014-05-13 12:46:10,548::clientIF::126::vds::(contEIOVms) vmContainerLock acquired >Thread-338::DEBUG::2014-05-13 12:46:10,564::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [249764c5] >Thread-338::DEBUG::2014-05-13 12:46:10,565::task::595::TaskManager.Task::(_updateState) Task=`d492f22b-8934-4141-9821-924bd6108856`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:46:10,565::logUtils::44::dispatcher::(wrapper) Run and protect: getFileStats(sdUUID='208946c2-5db1-4ef4-80b1-9aeb37f2905e', pattern='*.iso', caseSensitive=False, options=None) >Thread-338::DEBUG::2014-05-13 12:46:10,565::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e`ReqID=`6413c84b-eb79-4b63-8dc8-8c5376938a1c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2257' at 'getFileStats' >Thread-338::DEBUG::2014-05-13 12:46:10,566::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:46:10,566::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:46:10,566::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e`ReqID=`6413c84b-eb79-4b63-8dc8-8c5376938a1c`::Granted request >Thread-338::DEBUG::2014-05-13 12:46:10,566::task::827::TaskManager.Task::(resourceAcquired) Task=`d492f22b-8934-4141-9821-924bd6108856`::_resourcesAcquired: Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e (shared) >Thread-338::DEBUG::2014-05-13 12:46:10,566::task::990::TaskManager.Task::(_decref) Task=`d492f22b-8934-4141-9821-924bd6108856`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:46:10,572::logUtils::47::dispatcher::(wrapper) Run and protect: getFileStats, Return response: {'fileStats': {'rhev-tools-setup.iso': {'status': 0, 'ctime': '1399884928.91', 'size': '326424576'}, 'virtio-win.iso': {'status': 0, 'ctime': '1399884928.91', 'size': '97478656'}}} >Thread-338::DEBUG::2014-05-13 12:46:10,572::task::1185::TaskManager.Task::(prepare) Task=`d492f22b-8934-4141-9821-924bd6108856`::finished: {'fileStats': {'rhev-tools-setup.iso': {'status': 0, 'ctime': '1399884928.91', 'size': '326424576'}, 'virtio-win.iso': {'status': 0, 'ctime': '1399884928.91', 'size': '97478656'}}} >Thread-338::DEBUG::2014-05-13 12:46:10,572::task::595::TaskManager.Task::(_updateState) Task=`d492f22b-8934-4141-9821-924bd6108856`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:46:10,572::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e': < ResourceRef 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:46:10,573::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:46:10,573::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e' >Thread-338::DEBUG::2014-05-13 12:46:10,573::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:46:10,573::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:46:10,573::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:46:10,574::task::990::TaskManager.Task::(_decref) Task=`d492f22b-8934-4141-9821-924bd6108856`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:46:10,618::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [249764c5] >Thread-338::DEBUG::2014-05-13 12:46:10,618::task::595::TaskManager.Task::(_updateState) Task=`b58690f0-24f6-41c0-b125-3481431473ed`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:46:10,618::logUtils::44::dispatcher::(wrapper) Run and protect: getFileStats(sdUUID='208946c2-5db1-4ef4-80b1-9aeb37f2905e', pattern='*.vfd', caseSensitive=False, options=None) >Thread-338::DEBUG::2014-05-13 12:46:10,619::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e`ReqID=`6e3f050f-85f0-48a7-85a4-b5791a46cda7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2257' at 'getFileStats' >Thread-338::DEBUG::2014-05-13 12:46:10,619::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:46:10,619::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:46:10,619::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e`ReqID=`6e3f050f-85f0-48a7-85a4-b5791a46cda7`::Granted request >Thread-338::DEBUG::2014-05-13 12:46:10,620::task::827::TaskManager.Task::(resourceAcquired) Task=`b58690f0-24f6-41c0-b125-3481431473ed`::_resourcesAcquired: Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e (shared) >Thread-338::DEBUG::2014-05-13 12:46:10,620::task::990::TaskManager.Task::(_decref) Task=`b58690f0-24f6-41c0-b125-3481431473ed`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:46:10,623::logUtils::47::dispatcher::(wrapper) Run and protect: getFileStats, Return response: {'fileStats': {'virtio-win_x86.vfd': {'status': 0, 'ctime': '1399884928.91', 'size': '2949120'}, 'virtio-win_amd64.vfd': {'status': 0, 'ctime': '1399884928.91', 'size': '2949120'}}} >Thread-338::DEBUG::2014-05-13 12:46:10,624::task::1185::TaskManager.Task::(prepare) Task=`b58690f0-24f6-41c0-b125-3481431473ed`::finished: {'fileStats': {'virtio-win_x86.vfd': {'status': 0, 'ctime': '1399884928.91', 'size': '2949120'}, 'virtio-win_amd64.vfd': {'status': 0, 'ctime': '1399884928.91', 'size': '2949120'}}} >Thread-338::DEBUG::2014-05-13 12:46:10,624::task::595::TaskManager.Task::(_updateState) Task=`b58690f0-24f6-41c0-b125-3481431473ed`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:46:10,624::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e': < ResourceRef 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:46:10,624::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:46:10,624::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e' >Thread-338::DEBUG::2014-05-13 12:46:10,625::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:46:10,625::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:46:10,625::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.208946c2-5db1-4ef4-80b1-9aeb37f2905e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:46:10,625::task::990::TaskManager.Task::(_decref) Task=`b58690f0-24f6-41c0-b125-3481431473ed`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:46:11,268::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-344::DEBUG::2014-05-13 12:46:12,136::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:46:12,144::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000455777 s, 764 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:46:12,179::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:12,258::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0367592 s, 27.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:46:13,165::task::595::TaskManager.Task::(_updateState) Task=`2f88a8e5-1a56-4703-9166-04bdf7a6eaad`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:46:13,165::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:46:13,165::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000335891', 'lastCheck': '2.6', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000713713', 'lastCheck': '6.7', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000455777', 'lastCheck': '1.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:46:13,165::task::1185::TaskManager.Task::(prepare) Task=`2f88a8e5-1a56-4703-9166-04bdf7a6eaad`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000335891', 'lastCheck': '2.6', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000713713', 'lastCheck': '6.7', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000455777', 'lastCheck': '1.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:46:13,166::task::595::TaskManager.Task::(_updateState) Task=`2f88a8e5-1a56-4703-9166-04bdf7a6eaad`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:46:13,166::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:46:13,166::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:46:13,166::task::990::TaskManager.Task::(_decref) Task=`2f88a8e5-1a56-4703-9166-04bdf7a6eaad`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:46:14,272::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:46:14,289::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:46:14,349::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.032248 s, 31.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:46:16,362::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:16,438::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319513 s, 32.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:46:16,453::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:46:16,454::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 12:46:16,710::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:46:16,712::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:46:16,716::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:46:16,724::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000558887 s, 7.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:46:17,305::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:46:18,451::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:18,533::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0376671 s, 27.2 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:46:19,444::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1c80df6] >Thread-66::DEBUG::2014-05-13 12:46:19,444::task::595::TaskManager.Task::(_updateState) Task=`8c22970e-8f9b-4a9e-8fbc-073d0b1a89de`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:46:19,444::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:46:19,445::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:46:19,445::task::1185::TaskManager.Task::(prepare) Task=`8c22970e-8f9b-4a9e-8fbc-073d0b1a89de`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:46:19,445::task::595::TaskManager.Task::(_updateState) Task=`8c22970e-8f9b-4a9e-8fbc-073d0b1a89de`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:46:19,445::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:46:19,445::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:46:19,445::task::990::TaskManager.Task::(_decref) Task=`8c22970e-8f9b-4a9e-8fbc-073d0b1a89de`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:46:19,450::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1c80df6] >Thread-338::DEBUG::2014-05-13 12:46:19,450::task::595::TaskManager.Task::(_updateState) Task=`53a99c8c-1f6e-4170-b6d3-1eb8311d92d9`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:46:19,450::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:46:19,451::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`31372927-9811-49a3-bdd1-5226dbf859fc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:46:19,451::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:46:19,451::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:46:19,451::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`31372927-9811-49a3-bdd1-5226dbf859fc`::Granted request >Thread-338::DEBUG::2014-05-13 12:46:19,451::task::827::TaskManager.Task::(resourceAcquired) Task=`53a99c8c-1f6e-4170-b6d3-1eb8311d92d9`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:46:19,452::task::990::TaskManager.Task::(_decref) Task=`53a99c8c-1f6e-4170-b6d3-1eb8311d92d9`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:46:19,452::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:46:19,453::task::1185::TaskManager.Task::(prepare) Task=`53a99c8c-1f6e-4170-b6d3-1eb8311d92d9`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:46:19,453::task::595::TaskManager.Task::(_updateState) Task=`53a99c8c-1f6e-4170-b6d3-1eb8311d92d9`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:46:19,453::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:46:19,453::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:46:19,453::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:46:19,453::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:46:19,454::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:46:19,454::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:46:19,454::task::990::TaskManager.Task::(_decref) Task=`53a99c8c-1f6e-4170-b6d3-1eb8311d92d9`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:46:20,323::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:46:20,545::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-357::DEBUG::2014-05-13 12:46:20,554::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:46:20,564::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000498177 s, 785 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:46:20,640::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0496375 s, 20.6 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:46:22,154::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:46:22,162::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000354188 s, 983 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:46:22,653::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:22,737::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0396409 s, 25.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:46:23,342::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:46:24,750::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:24,827::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0328212 s, 31.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:46:26,360::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:46:26,733::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:46:26,741::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000687231 s, 6.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:46:26,840::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:26,921::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359594 s, 28.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:46:28,296::task::595::TaskManager.Task::(_updateState) Task=`bf4444e0-c78b-498d-ab92-b559dabb51fe`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:46:28,296::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:46:28,296::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000498177', 'lastCheck': '7.7', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000687231', 'lastCheck': '1.5', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000354188', 'lastCheck': '6.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:46:28,297::task::1185::TaskManager.Task::(prepare) Task=`bf4444e0-c78b-498d-ab92-b559dabb51fe`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000498177', 'lastCheck': '7.7', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000687231', 'lastCheck': '1.5', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000354188', 'lastCheck': '6.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:46:28,297::task::595::TaskManager.Task::(_updateState) Task=`bf4444e0-c78b-498d-ab92-b559dabb51fe`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:46:28,297::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:46:28,297::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:46:28,297::task::990::TaskManager.Task::(_decref) Task=`bf4444e0-c78b-498d-ab92-b559dabb51fe`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:46:28,935::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:29,012::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320061 s, 32.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:46:29,381::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:46:29,499::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6623fb10] >Thread-66::DEBUG::2014-05-13 12:46:29,500::task::595::TaskManager.Task::(_updateState) Task=`e8a11092-a7ad-4df2-8e46-6431ad0344bd`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:46:29,500::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:46:29,500::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:46:29,501::task::1185::TaskManager.Task::(prepare) Task=`e8a11092-a7ad-4df2-8e46-6431ad0344bd`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:46:29,501::task::595::TaskManager.Task::(_updateState) Task=`e8a11092-a7ad-4df2-8e46-6431ad0344bd`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:46:29,501::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:46:29,501::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:46:29,501::task::990::TaskManager.Task::(_decref) Task=`e8a11092-a7ad-4df2-8e46-6431ad0344bd`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:46:29,506::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6623fb10] >Thread-338::DEBUG::2014-05-13 12:46:29,506::task::595::TaskManager.Task::(_updateState) Task=`221d3809-43a6-4877-9586-f7e1263da75c`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:46:29,506::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:46:29,507::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4e25a48e-ce28-4b08-9731-b123a45c9d78`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:46:29,507::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:46:29,507::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:46:29,507::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4e25a48e-ce28-4b08-9731-b123a45c9d78`::Granted request >Thread-338::DEBUG::2014-05-13 12:46:29,508::task::827::TaskManager.Task::(resourceAcquired) Task=`221d3809-43a6-4877-9586-f7e1263da75c`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:46:29,508::task::990::TaskManager.Task::(_decref) Task=`221d3809-43a6-4877-9586-f7e1263da75c`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:46:29,509::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:46:29,509::task::1185::TaskManager.Task::(prepare) Task=`221d3809-43a6-4877-9586-f7e1263da75c`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:46:29,509::task::595::TaskManager.Task::(_updateState) Task=`221d3809-43a6-4877-9586-f7e1263da75c`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:46:29,509::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:46:29,509::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:46:29,510::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:46:29,510::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:46:29,510::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:46:29,510::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:46:29,510::task::990::TaskManager.Task::(_decref) Task=`221d3809-43a6-4877-9586-f7e1263da75c`::ref 0 aborting False >Thread-357::DEBUG::2014-05-13 12:46:30,578::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:46:30,587::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000571013 s, 685 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:46:31,025::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:31,098::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311896 s, 32.8 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:46:32,173::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:46:32,182::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000398266 s, 874 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:46:32,399::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:46:33,111::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:33,195::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0398784 s, 25.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:46:35,208::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:35,294::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0431787 s, 23.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:46:35,418::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:46:36,751::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:46:36,759::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000712878 s, 5.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:46:37,307::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:37,409::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0434176 s, 23.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:46:38,436::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:46:39,424::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:39,518::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0415545 s, 24.6 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:46:39,544::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:46:39,545::task::595::TaskManager.Task::(_updateState) Task=`45574b3d-8157-43cc-b8a9-974f2d457055`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:46:39,545::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:46:39,545::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:46:39,546::task::1185::TaskManager.Task::(prepare) Task=`45574b3d-8157-43cc-b8a9-974f2d457055`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:46:39,546::task::595::TaskManager.Task::(_updateState) Task=`45574b3d-8157-43cc-b8a9-974f2d457055`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:46:39,546::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:46:39,546::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:46:39,546::task::990::TaskManager.Task::(_decref) Task=`45574b3d-8157-43cc-b8a9-974f2d457055`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:46:39,551::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:46:39,551::task::595::TaskManager.Task::(_updateState) Task=`988fd805-5215-4b68-bb98-225742f10ebe`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:46:39,551::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:46:39,552::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a857442b-37a3-406e-a4ca-9cf98c36a6af`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:46:39,552::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:46:39,552::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:46:39,552::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a857442b-37a3-406e-a4ca-9cf98c36a6af`::Granted request >Thread-338::DEBUG::2014-05-13 12:46:39,553::task::827::TaskManager.Task::(resourceAcquired) Task=`988fd805-5215-4b68-bb98-225742f10ebe`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:46:39,553::task::990::TaskManager.Task::(_decref) Task=`988fd805-5215-4b68-bb98-225742f10ebe`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:46:39,554::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:46:39,554::task::1185::TaskManager.Task::(prepare) Task=`988fd805-5215-4b68-bb98-225742f10ebe`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:46:39,554::task::595::TaskManager.Task::(_updateState) Task=`988fd805-5215-4b68-bb98-225742f10ebe`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:46:39,554::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:46:39,554::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:46:39,555::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:46:39,555::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:46:39,555::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:46:39,555::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:46:39,555::task::990::TaskManager.Task::(_decref) Task=`988fd805-5215-4b68-bb98-225742f10ebe`::ref 0 aborting False >Thread-357::DEBUG::2014-05-13 12:46:40,597::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:46:40,606::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000652753 s, 599 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:46:41,454::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:46:41,532::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:41,615::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0399584 s, 25.6 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:46:42,194::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:46:42,203::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000346295 s, 1.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:46:43,437::task::595::TaskManager.Task::(_updateState) Task=`cced9103-8711-4db7-96dc-084c486a29c7`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:46:43,437::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:46:43,438::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000652753', 'lastCheck': '2.8', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000712878', 'lastCheck': '6.7', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000346295', 'lastCheck': '1.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:46:43,438::task::1185::TaskManager.Task::(prepare) Task=`cced9103-8711-4db7-96dc-084c486a29c7`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000652753', 'lastCheck': '2.8', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000712878', 'lastCheck': '6.7', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000346295', 'lastCheck': '1.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:46:43,438::task::595::TaskManager.Task::(_updateState) Task=`cced9103-8711-4db7-96dc-084c486a29c7`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:46:43,438::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:46:43,438::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:46:43,439::task::990::TaskManager.Task::(_decref) Task=`cced9103-8711-4db7-96dc-084c486a29c7`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:46:43,628::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:43,703::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0324944 s, 31.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:46:44,477::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:46:45,716::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:45,793::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0324968 s, 31.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:46:46,767::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:46:46,775::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000756511 s, 5.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:46:47,495::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:46:47,806::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:47,896::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0476963 s, 21.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:46:49,590::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:46:49,590::task::595::TaskManager.Task::(_updateState) Task=`bf7bb28c-89d8-4817-992c-a8d2d76cdf60`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:46:49,590::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:46:49,591::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:46:49,591::task::1185::TaskManager.Task::(prepare) Task=`bf7bb28c-89d8-4817-992c-a8d2d76cdf60`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:46:49,591::task::595::TaskManager.Task::(_updateState) Task=`bf7bb28c-89d8-4817-992c-a8d2d76cdf60`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:46:49,591::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:46:49,591::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:46:49,591::task::990::TaskManager.Task::(_decref) Task=`bf7bb28c-89d8-4817-992c-a8d2d76cdf60`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:46:49,596::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:46:49,596::task::595::TaskManager.Task::(_updateState) Task=`9a721018-e9f0-4b28-bc1d-4c7a526e7a1d`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:46:49,596::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:46:49,596::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9da701e7-17c6-4ddc-a707-04efcae011ad`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:46:49,597::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:46:49,597::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:46:49,597::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9da701e7-17c6-4ddc-a707-04efcae011ad`::Granted request >Thread-338::DEBUG::2014-05-13 12:46:49,597::task::827::TaskManager.Task::(resourceAcquired) Task=`9a721018-e9f0-4b28-bc1d-4c7a526e7a1d`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:46:49,597::task::990::TaskManager.Task::(_decref) Task=`9a721018-e9f0-4b28-bc1d-4c7a526e7a1d`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:46:49,598::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:46:49,598::task::1185::TaskManager.Task::(prepare) Task=`9a721018-e9f0-4b28-bc1d-4c7a526e7a1d`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:46:49,598::task::595::TaskManager.Task::(_updateState) Task=`9a721018-e9f0-4b28-bc1d-4c7a526e7a1d`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:46:49,599::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:46:49,599::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:46:49,599::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:46:49,599::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:46:49,599::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:46:49,599::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:46:49,600::task::990::TaskManager.Task::(_decref) Task=`9a721018-e9f0-4b28-bc1d-4c7a526e7a1d`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:46:49,909::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:49,987::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0326074 s, 31.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:46:50,522::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-357::DEBUG::2014-05-13 12:46:50,618::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:46:50,626::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000409647 s, 954 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:46:52,000::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:52,075::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321703 s, 31.8 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:46:52,215::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:46:52,224::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000405663 s, 858 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:46:53,541::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:46:54,088::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:54,169::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.036298 s, 28.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:46:56,181::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:56,258::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0329794 s, 31.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:46:56,563::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:46:56,784::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:46:56,792::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000797182 s, 5.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:46:58,275::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:46:58,348::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317028 s, 32.3 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:46:58,574::task::595::TaskManager.Task::(_updateState) Task=`e3b39572-7ccd-4031-af9a-72c40830a517`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:46:58,575::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:46:58,575::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000409647', 'lastCheck': '7.9', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000797182', 'lastCheck': '1.8', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000405663', 'lastCheck': '6.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:46:58,575::task::1185::TaskManager.Task::(prepare) Task=`e3b39572-7ccd-4031-af9a-72c40830a517`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000409647', 'lastCheck': '7.9', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000797182', 'lastCheck': '1.8', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000405663', 'lastCheck': '6.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:46:58,575::task::595::TaskManager.Task::(_updateState) Task=`e3b39572-7ccd-4031-af9a-72c40830a517`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:46:58,576::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:46:58,576::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:46:58,576::task::990::TaskManager.Task::(_decref) Task=`e3b39572-7ccd-4031-af9a-72c40830a517`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:46:59,582::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:46:59,635::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:46:59,636::task::595::TaskManager.Task::(_updateState) Task=`d1bcd892-96a9-439e-8e13-1357ce6285f2`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:46:59,636::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:46:59,636::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:46:59,636::task::1185::TaskManager.Task::(prepare) Task=`d1bcd892-96a9-439e-8e13-1357ce6285f2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:46:59,637::task::595::TaskManager.Task::(_updateState) Task=`d1bcd892-96a9-439e-8e13-1357ce6285f2`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:46:59,637::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:46:59,637::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:46:59,637::task::990::TaskManager.Task::(_decref) Task=`d1bcd892-96a9-439e-8e13-1357ce6285f2`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:46:59,641::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:46:59,642::task::595::TaskManager.Task::(_updateState) Task=`c6cf6d52-90e4-434b-b665-32e4b78d71c3`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:46:59,642::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:46:59,642::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c5ec609f-ca60-4700-be6d-53f8a0298e53`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:46:59,643::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:46:59,643::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:46:59,643::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c5ec609f-ca60-4700-be6d-53f8a0298e53`::Granted request >Thread-338::DEBUG::2014-05-13 12:46:59,643::task::827::TaskManager.Task::(resourceAcquired) Task=`c6cf6d52-90e4-434b-b665-32e4b78d71c3`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:46:59,644::task::990::TaskManager.Task::(_decref) Task=`c6cf6d52-90e4-434b-b665-32e4b78d71c3`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:46:59,644::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:46:59,645::task::1185::TaskManager.Task::(prepare) Task=`c6cf6d52-90e4-434b-b665-32e4b78d71c3`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:46:59,645::task::595::TaskManager.Task::(_updateState) Task=`c6cf6d52-90e4-434b-b665-32e4b78d71c3`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:46:59,645::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:46:59,645::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:46:59,645::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:46:59,646::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:46:59,646::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:46:59,646::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:46:59,646::task::990::TaskManager.Task::(_decref) Task=`c6cf6d52-90e4-434b-b665-32e4b78d71c3`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:47:00,361::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:00,442::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373477 s, 27.4 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:47:00,638::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:47:00,647::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000398777 s, 980 kB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:47:02,238::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:47:02,247::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000353328 s, 985 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:47:02,455::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:02,549::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0508183 s, 20.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:47:02,599::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:47:04,561::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:04,639::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0346478 s, 29.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:47:05,617::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:47:06,652::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:06,727::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0389997 s, 26.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:47:06,800::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:47:06,808::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00078852 s, 5.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:47:08,634::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:47:08,736::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:08,820::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0416931 s, 24.6 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:47:09,682::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2555e502] >Thread-66::DEBUG::2014-05-13 12:47:09,683::task::595::TaskManager.Task::(_updateState) Task=`1028a4ca-14ea-44a6-a501-45705ad1bb5d`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:47:09,683::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:47:09,683::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:47:09,684::task::1185::TaskManager.Task::(prepare) Task=`1028a4ca-14ea-44a6-a501-45705ad1bb5d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:47:09,684::task::595::TaskManager.Task::(_updateState) Task=`1028a4ca-14ea-44a6-a501-45705ad1bb5d`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:47:09,684::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:47:09,684::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:47:09,684::task::990::TaskManager.Task::(_decref) Task=`1028a4ca-14ea-44a6-a501-45705ad1bb5d`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:47:09,690::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2555e502] >Thread-338::DEBUG::2014-05-13 12:47:09,690::task::595::TaskManager.Task::(_updateState) Task=`ae516c84-e058-4a26-afe4-00ad487d3b3a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:47:09,690::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:47:09,691::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`11dfbe1c-1c46-4be8-92e6-b3182c9f4452`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:47:09,691::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:47:09,691::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:47:09,691::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`11dfbe1c-1c46-4be8-92e6-b3182c9f4452`::Granted request >Thread-338::DEBUG::2014-05-13 12:47:09,692::task::827::TaskManager.Task::(resourceAcquired) Task=`ae516c84-e058-4a26-afe4-00ad487d3b3a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:47:09,692::task::990::TaskManager.Task::(_decref) Task=`ae516c84-e058-4a26-afe4-00ad487d3b3a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:47:09,693::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:47:09,693::task::1185::TaskManager.Task::(prepare) Task=`ae516c84-e058-4a26-afe4-00ad487d3b3a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:47:09,693::task::595::TaskManager.Task::(_updateState) Task=`ae516c84-e058-4a26-afe4-00ad487d3b3a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:47:09,693::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:47:09,693::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:47:09,694::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:47:09,694::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:47:09,694::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:47:09,694::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:47:09,694::task::990::TaskManager.Task::(_decref) Task=`ae516c84-e058-4a26-afe4-00ad487d3b3a`::ref 0 aborting False >Thread-357::DEBUG::2014-05-13 12:47:10,662::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:47:10,671::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000458215 s, 853 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:47:10,833::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:10,911::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.03602 s, 28.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:47:11,662::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-344::DEBUG::2014-05-13 12:47:12,259::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:47:12,267::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000436831 s, 797 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:47:12,924::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:13,000::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322144 s, 31.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:47:13,717::task::595::TaskManager.Task::(_updateState) Task=`cf01d9f1-8594-4a44-9f82-3bea09290cb4`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:47:13,718::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:47:13,718::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000458215', 'lastCheck': '3.0', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00078852', 'lastCheck': '6.9', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000436831', 'lastCheck': '1.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:47:13,718::task::1185::TaskManager.Task::(prepare) Task=`cf01d9f1-8594-4a44-9f82-3bea09290cb4`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000458215', 'lastCheck': '3.0', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00078852', 'lastCheck': '6.9', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000436831', 'lastCheck': '1.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:47:13,718::task::595::TaskManager.Task::(_updateState) Task=`cf01d9f1-8594-4a44-9f82-3bea09290cb4`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:47:13,719::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:47:13,719::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:47:13,719::task::990::TaskManager.Task::(_decref) Task=`cf01d9f1-8594-4a44-9f82-3bea09290cb4`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:47:14,680::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:47:15,013::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:15,092::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357359 s, 28.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:47:16,816::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:47:16,826::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00199488 s, 2.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:47:17,105::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:17,195::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0461389 s, 22.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:47:17,698::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:47:19,208::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:19,288::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365741 s, 28.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:47:19,730::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:47:19,730::task::595::TaskManager.Task::(_updateState) Task=`f8953533-1463-4d35-b404-4075be290a6c`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:47:19,730::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:47:19,731::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:47:19,731::task::1185::TaskManager.Task::(prepare) Task=`f8953533-1463-4d35-b404-4075be290a6c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:47:19,731::task::595::TaskManager.Task::(_updateState) Task=`f8953533-1463-4d35-b404-4075be290a6c`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:47:19,731::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:47:19,731::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:47:19,732::task::990::TaskManager.Task::(_decref) Task=`f8953533-1463-4d35-b404-4075be290a6c`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:47:19,738::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:47:19,738::task::595::TaskManager.Task::(_updateState) Task=`2ca621bd-a3b8-4961-bc00-b312aa708058`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:47:19,739::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:47:19,739::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`18c325d7-07f0-4cb4-bf28-bda56f6c651c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:47:19,739::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:47:19,739::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:47:19,740::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`18c325d7-07f0-4cb4-bf28-bda56f6c651c`::Granted request >Thread-338::DEBUG::2014-05-13 12:47:19,740::task::827::TaskManager.Task::(resourceAcquired) Task=`2ca621bd-a3b8-4961-bc00-b312aa708058`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:47:19,740::task::990::TaskManager.Task::(_decref) Task=`2ca621bd-a3b8-4961-bc00-b312aa708058`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:47:19,741::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:47:19,741::task::1185::TaskManager.Task::(prepare) Task=`2ca621bd-a3b8-4961-bc00-b312aa708058`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:47:19,741::task::595::TaskManager.Task::(_updateState) Task=`2ca621bd-a3b8-4961-bc00-b312aa708058`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:47:19,742::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:47:19,742::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:47:19,742::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:47:19,742::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:47:19,742::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:47:19,743::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:47:19,743::task::990::TaskManager.Task::(_decref) Task=`2ca621bd-a3b8-4961-bc00-b312aa708058`::ref 0 aborting False >Thread-357::DEBUG::2014-05-13 12:47:20,682::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:47:20,691::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000382217 s, 1.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:47:20,716::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:47:21,301::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:21,382::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037621 s, 27.2 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:47:22,278::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:47:22,286::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000399779 s, 870 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:47:23,395::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:23,478::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037783 s, 27.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:47:23,734::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:47:25,491::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:25,571::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357898 s, 28.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:47:26,752::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:47:26,834::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:47:26,843::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.0013541 s, 3.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:47:27,585::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:27,670::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0398357 s, 25.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:47:28,851::task::595::TaskManager.Task::(_updateState) Task=`582f97b3-d2d1-457f-856c-10157fcc03d3`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:47:28,852::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:47:28,852::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000382217', 'lastCheck': '8.2', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0013541', 'lastCheck': '2.0', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000399779', 'lastCheck': '6.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:47:28,852::task::1185::TaskManager.Task::(prepare) Task=`582f97b3-d2d1-457f-856c-10157fcc03d3`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000382217', 'lastCheck': '8.2', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0013541', 'lastCheck': '2.0', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000399779', 'lastCheck': '6.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:47:28,852::task::595::TaskManager.Task::(_updateState) Task=`582f97b3-d2d1-457f-856c-10157fcc03d3`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:47:28,852::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:47:28,853::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:47:28,853::task::990::TaskManager.Task::(_decref) Task=`582f97b3-d2d1-457f-856c-10157fcc03d3`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:47:29,683::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:47:29,771::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:47:29,778::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:47:29,779::task::595::TaskManager.Task::(_updateState) Task=`ef29bb5d-0980-4d40-a4b9-a50d277d57f1`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:47:29,779::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:47:29,779::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:47:29,779::task::1185::TaskManager.Task::(prepare) Task=`ef29bb5d-0980-4d40-a4b9-a50d277d57f1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:47:29,780::task::595::TaskManager.Task::(_updateState) Task=`ef29bb5d-0980-4d40-a4b9-a50d277d57f1`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:47:29,780::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:47:29,780::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:47:29,780::task::990::TaskManager.Task::(_decref) Task=`ef29bb5d-0980-4d40-a4b9-a50d277d57f1`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:47:29,784::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:47:29,785::task::595::TaskManager.Task::(_updateState) Task=`9b15908c-8049-4eee-a474-2b95757754be`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:47:29,785::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:47:29,785::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c56ae70d-1fe2-4eec-a133-5e83faaddc6c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:47:29,785::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:47:29,786::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:47:29,786::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c56ae70d-1fe2-4eec-a133-5e83faaddc6c`::Granted request >Thread-338::DEBUG::2014-05-13 12:47:29,786::task::827::TaskManager.Task::(resourceAcquired) Task=`9b15908c-8049-4eee-a474-2b95757754be`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:47:29,786::task::990::TaskManager.Task::(_decref) Task=`9b15908c-8049-4eee-a474-2b95757754be`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:47:29,787::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:47:29,787::task::1185::TaskManager.Task::(prepare) Task=`9b15908c-8049-4eee-a474-2b95757754be`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:47:29,788::task::595::TaskManager.Task::(_updateState) Task=`9b15908c-8049-4eee-a474-2b95757754be`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:47:29,788::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:47:29,788::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:47:29,788::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:47:29,788::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:47:29,788::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:47:29,789::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:47:29,789::task::990::TaskManager.Task::(_decref) Task=`9b15908c-8049-4eee-a474-2b95757754be`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:47:30,099::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.374118 s, 2.7 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:47:30,702::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:47:30,711::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000491902 s, 795 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:47:32,112::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:32,186::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311477 s, 32.9 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:47:32,297::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:47:32,305::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000399638 s, 871 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:47:32,788::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:47:34,199::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:34,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382411 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:47:35,808::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:47:36,292::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:36,374::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0371517 s, 27.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:47:36,850::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:47:36,858::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000627812 s, 6.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:47:38,388::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:38,767::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.335743 s, 3.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:47:38,830::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:47:39,823::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:47:39,823::task::595::TaskManager.Task::(_updateState) Task=`6cd0ec33-910b-4fff-a9fb-1ac28662464e`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:47:39,824::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:47:39,824::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:47:39,824::task::1185::TaskManager.Task::(prepare) Task=`6cd0ec33-910b-4fff-a9fb-1ac28662464e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:47:39,824::task::595::TaskManager.Task::(_updateState) Task=`6cd0ec33-910b-4fff-a9fb-1ac28662464e`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:47:39,824::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:47:39,824::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:47:39,825::task::990::TaskManager.Task::(_decref) Task=`6cd0ec33-910b-4fff-a9fb-1ac28662464e`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:47:39,829::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:47:39,829::task::595::TaskManager.Task::(_updateState) Task=`31057ec5-0f85-4a59-846e-a10d60bb2693`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:47:39,829::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:47:39,830::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d0514c0f-3e37-4994-96fd-fe6e7245c3e9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:47:39,830::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:47:39,830::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:47:39,830::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d0514c0f-3e37-4994-96fd-fe6e7245c3e9`::Granted request >Thread-338::DEBUG::2014-05-13 12:47:39,831::task::827::TaskManager.Task::(resourceAcquired) Task=`31057ec5-0f85-4a59-846e-a10d60bb2693`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:47:39,831::task::990::TaskManager.Task::(_decref) Task=`31057ec5-0f85-4a59-846e-a10d60bb2693`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:47:39,832::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:47:39,832::task::1185::TaskManager.Task::(prepare) Task=`31057ec5-0f85-4a59-846e-a10d60bb2693`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:47:39,832::task::595::TaskManager.Task::(_updateState) Task=`31057ec5-0f85-4a59-846e-a10d60bb2693`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:47:39,832::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:47:39,832::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:47:39,833::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:47:39,833::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:47:39,833::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:47:39,833::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:47:39,833::task::990::TaskManager.Task::(_decref) Task=`31057ec5-0f85-4a59-846e-a10d60bb2693`::ref 0 aborting False >Thread-357::DEBUG::2014-05-13 12:47:40,721::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:47:40,730::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000722838 s, 541 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:47:40,780::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:40,865::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0389896 s, 26.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:47:41,848::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-344::DEBUG::2014-05-13 12:47:42,317::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:47:42,325::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000390734 s, 891 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:47:42,879::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:42,961::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0388106 s, 26.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:47:43,987::task::595::TaskManager.Task::(_updateState) Task=`9e925a6f-ef59-4431-962a-2287d0fb2486`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:47:43,987::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:47:43,988::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000722838', 'lastCheck': '3.3', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000627812', 'lastCheck': '7.1', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000390734', 'lastCheck': '1.7', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:47:43,988::task::1185::TaskManager.Task::(prepare) Task=`9e925a6f-ef59-4431-962a-2287d0fb2486`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000722838', 'lastCheck': '3.3', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000627812', 'lastCheck': '7.1', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000390734', 'lastCheck': '1.7', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:47:43,988::task::595::TaskManager.Task::(_updateState) Task=`9e925a6f-ef59-4431-962a-2287d0fb2486`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:47:43,988::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:47:43,988::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:47:43,988::task::990::TaskManager.Task::(_decref) Task=`9e925a6f-ef59-4431-962a-2287d0fb2486`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:47:44,867::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:47:44,975::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:45,058::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383743 s, 26.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:47:46,866::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:47:46,874::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000642287 s, 6.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:47:47,072::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:47,152::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0370343 s, 27.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:47:47,885::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:47:49,165::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:49,241::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312205 s, 32.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:47:49,869::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:47:49,911::task::595::TaskManager.Task::(_updateState) Task=`a3f5f65d-d089-45bc-963e-54b07b8a83df`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:47:49,911::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:47:49,912::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:47:49,912::task::1185::TaskManager.Task::(prepare) Task=`a3f5f65d-d089-45bc-963e-54b07b8a83df`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:47:49,912::task::595::TaskManager.Task::(_updateState) Task=`a3f5f65d-d089-45bc-963e-54b07b8a83df`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:47:49,912::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:47:49,912::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:47:49,912::task::990::TaskManager.Task::(_decref) Task=`a3f5f65d-d089-45bc-963e-54b07b8a83df`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:47:49,917::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:47:49,918::task::595::TaskManager.Task::(_updateState) Task=`32d24c2a-1cf8-4dc4-9200-642968ebd806`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:47:49,918::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:47:49,918::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6bf8b269-371c-42a2-b668-889e17355b85`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:47:49,918::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:47:49,919::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:47:49,919::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6bf8b269-371c-42a2-b668-889e17355b85`::Granted request >Thread-338::DEBUG::2014-05-13 12:47:49,919::task::827::TaskManager.Task::(resourceAcquired) Task=`32d24c2a-1cf8-4dc4-9200-642968ebd806`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:47:49,919::task::990::TaskManager.Task::(_decref) Task=`32d24c2a-1cf8-4dc4-9200-642968ebd806`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:47:49,920::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:47:49,921::task::1185::TaskManager.Task::(prepare) Task=`32d24c2a-1cf8-4dc4-9200-642968ebd806`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:47:49,921::task::595::TaskManager.Task::(_updateState) Task=`32d24c2a-1cf8-4dc4-9200-642968ebd806`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:47:49,921::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:47:49,921::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:47:49,921::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:47:49,922::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:47:49,922::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:47:49,922::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:47:49,922::task::990::TaskManager.Task::(_decref) Task=`32d24c2a-1cf8-4dc4-9200-642968ebd806`::ref 0 aborting False >Thread-357::DEBUG::2014-05-13 12:47:50,742::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:47:50,751::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000507993 s, 770 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:47:50,904::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:47:51,254::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:51,337::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0392641 s, 26.1 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:47:52,335::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:47:52,344::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000411938 s, 845 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:47:53,350::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:53,442::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.049379 s, 20.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:47:53,925::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:47:55,455::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:55,532::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320535 s, 31.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:47:56,882::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:47:56,890::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000706848 s, 5.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:47:56,942::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:47:57,545::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:57,639::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0504019 s, 20.3 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:47:59,141::task::595::TaskManager.Task::(_updateState) Task=`71db8e30-db5d-43fe-98da-193f719aa01e`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:47:59,142::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:47:59,142::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000507993', 'lastCheck': '8.4', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000706848', 'lastCheck': '2.2', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000411938', 'lastCheck': '6.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:47:59,142::task::1185::TaskManager.Task::(prepare) Task=`71db8e30-db5d-43fe-98da-193f719aa01e`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000507993', 'lastCheck': '8.4', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000706848', 'lastCheck': '2.2', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000411938', 'lastCheck': '6.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:47:59,142::task::595::TaskManager.Task::(_updateState) Task=`71db8e30-db5d-43fe-98da-193f719aa01e`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:47:59,143::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:47:59,143::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:47:59,143::task::990::TaskManager.Task::(_decref) Task=`71db8e30-db5d-43fe-98da-193f719aa01e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:47:59,652::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:47:59,732::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360371 s, 28.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:47:59,959::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2ec76adc] >Thread-66::DEBUG::2014-05-13 12:47:59,959::task::595::TaskManager.Task::(_updateState) Task=`40622dec-f8cd-4289-9dcc-8d8a9b7a797f`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:47:59,959::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:47:59,960::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:47:59,960::task::1185::TaskManager.Task::(prepare) Task=`40622dec-f8cd-4289-9dcc-8d8a9b7a797f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:47:59,960::task::595::TaskManager.Task::(_updateState) Task=`40622dec-f8cd-4289-9dcc-8d8a9b7a797f`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:47:59,961::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:47:59,962::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >BindingXMLRPC::ERROR::2014-05-13 12:47:59,961::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:47:59,963::task::990::TaskManager.Task::(_decref) Task=`40622dec-f8cd-4289-9dcc-8d8a9b7a797f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:47:59,968::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2ec76adc] >Thread-338::DEBUG::2014-05-13 12:47:59,968::task::595::TaskManager.Task::(_updateState) Task=`972867f3-0a9a-4562-8012-8cf1c0a82644`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:47:59,969::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:47:59,969::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e593a84c-990c-4416-afd9-49aa5530e246`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:47:59,969::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:47:59,970::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:47:59,970::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e593a84c-990c-4416-afd9-49aa5530e246`::Granted request >Thread-338::DEBUG::2014-05-13 12:47:59,970::task::827::TaskManager.Task::(resourceAcquired) Task=`972867f3-0a9a-4562-8012-8cf1c0a82644`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:47:59,970::task::990::TaskManager.Task::(_decref) Task=`972867f3-0a9a-4562-8012-8cf1c0a82644`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:47:59,971::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:47:59,971::task::1185::TaskManager.Task::(prepare) Task=`972867f3-0a9a-4562-8012-8cf1c0a82644`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:47:59,972::task::595::TaskManager.Task::(_updateState) Task=`972867f3-0a9a-4562-8012-8cf1c0a82644`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:47:59,972::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:47:59,972::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:47:59,972::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:47:59,972::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:47:59,973::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:47:59,973::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:47:59,973::task::990::TaskManager.Task::(_decref) Task=`972867f3-0a9a-4562-8012-8cf1c0a82644`::ref 0 aborting False >Thread-357::DEBUG::2014-05-13 12:48:00,765::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:48:00,774::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000505864 s, 773 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:48:01,744::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:48:01,827::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0389378 s, 26.3 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:48:02,355::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:48:02,364::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000350796 s, 992 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:48:02,977::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:48:03,840::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:48:03,917::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321417 s, 31.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:48:05,930::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:48:06,000::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:48:06,341::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.367862 s, 2.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:48:06,898::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:48:06,906::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000690668 s, 5.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:48:08,354::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:48:08,433::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358978 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:48:09,019::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:48:10,009::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:48:10,009::task::595::TaskManager.Task::(_updateState) Task=`735a2547-1328-4955-861e-96e2e9036f26`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:48:10,009::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:48:10,010::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:48:10,010::task::1185::TaskManager.Task::(prepare) Task=`735a2547-1328-4955-861e-96e2e9036f26`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:48:10,010::task::595::TaskManager.Task::(_updateState) Task=`735a2547-1328-4955-861e-96e2e9036f26`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:48:10,010::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:48:10,010::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:48:10,011::task::990::TaskManager.Task::(_decref) Task=`735a2547-1328-4955-861e-96e2e9036f26`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:48:10,015::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:48:10,015::task::595::TaskManager.Task::(_updateState) Task=`7dc2a34d-3c5c-4a82-b490-a83284d8d8c5`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:48:10,015::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:48:10,016::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`116ede77-c1e6-4506-8867-9c982165603c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:48:10,016::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:48:10,016::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:48:10,016::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`116ede77-c1e6-4506-8867-9c982165603c`::Granted request >Thread-338::DEBUG::2014-05-13 12:48:10,016::task::827::TaskManager.Task::(resourceAcquired) Task=`7dc2a34d-3c5c-4a82-b490-a83284d8d8c5`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:48:10,017::task::990::TaskManager.Task::(_decref) Task=`7dc2a34d-3c5c-4a82-b490-a83284d8d8c5`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:48:10,017::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:48:10,017::task::1185::TaskManager.Task::(prepare) Task=`7dc2a34d-3c5c-4a82-b490-a83284d8d8c5`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:48:10,018::task::595::TaskManager.Task::(_updateState) Task=`7dc2a34d-3c5c-4a82-b490-a83284d8d8c5`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:48:10,018::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:48:10,018::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:48:10,018::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:48:10,018::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:48:10,019::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:48:10,019::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:48:10,019::task::990::TaskManager.Task::(_decref) Task=`7dc2a34d-3c5c-4a82-b490-a83284d8d8c5`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:48:10,446::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:48:10,520::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310147 s, 33.0 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:48:10,787::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:48:10,795::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000514011 s, 761 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:48:12,037::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-344::DEBUG::2014-05-13 12:48:12,366::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 572b0496-5eab-41cd-a976-4514f111898e >Thread-344::DEBUG::2014-05-13 12:48:12,374::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e >Thread-344::DEBUG::2014-05-13 12:48:12,375::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-344::DEBUG::2014-05-13 12:48:12,382::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=NFS_2', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', 'REMOTE_PATH=10.35.160.108:/RHEV/ogofen/3', 'ROLE=Regular', 'SDUUID=572b0496-5eab-41cd-a976-4514f111898e', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=06893a9d3c8506053131030a281fb9b9deb4efa9'] >Thread-344::DEBUG::2014-05-13 12:48:12,383::fileSD::609::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-344::INFO::2014-05-13 12:48:12,383::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 572b0496-5eab-41cd-a976-4514f111898e_imageNS already registered >Thread-344::INFO::2014-05-13 12:48:12,384::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 572b0496-5eab-41cd-a976-4514f111898e_volumeNS already registered >Thread-344::DEBUG::2014-05-13 12:48:12,393::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:48:12,401::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000413932 s, 841 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:48:12,533::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:48:12,617::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382824 s, 26.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:48:14,561::task::595::TaskManager.Task::(_updateState) Task=`9afa1ec3-4557-4399-9aa8-9b86b1710746`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:48:14,562::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:48:14,562::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000514011', 'lastCheck': '3.8', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000690668', 'lastCheck': '7.7', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000413932', 'lastCheck': '2.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:48:14,562::task::1185::TaskManager.Task::(prepare) Task=`9afa1ec3-4557-4399-9aa8-9b86b1710746`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000514011', 'lastCheck': '3.8', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000690668', 'lastCheck': '7.7', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000413932', 'lastCheck': '2.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:48:14,562::task::595::TaskManager.Task::(_updateState) Task=`9afa1ec3-4557-4399-9aa8-9b86b1710746`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:48:14,563::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:48:14,563::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:48:14,563::task::990::TaskManager.Task::(_decref) Task=`9afa1ec3-4557-4399-9aa8-9b86b1710746`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:48:14,630::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:48:14,709::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358722 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:48:15,054::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:48:16,722::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:48:16,796::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306875 s, 33.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:48:16,914::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:48:16,923::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000725505 s, 5.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:48:18,075::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:48:18,809::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:48:18,884::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322645 s, 31.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:48:20,063::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:48:20,064::task::595::TaskManager.Task::(_updateState) Task=`8e6b2279-c9b7-476e-85a0-3fcb4dbeb90d`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:48:20,064::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:48:20,064::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:48:20,064::task::1185::TaskManager.Task::(prepare) Task=`8e6b2279-c9b7-476e-85a0-3fcb4dbeb90d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:48:20,065::task::595::TaskManager.Task::(_updateState) Task=`8e6b2279-c9b7-476e-85a0-3fcb4dbeb90d`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:48:20,065::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:48:20,065::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:48:20,065::task::990::TaskManager.Task::(_decref) Task=`8e6b2279-c9b7-476e-85a0-3fcb4dbeb90d`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:48:20,069::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:48:20,070::task::595::TaskManager.Task::(_updateState) Task=`97492d73-2aac-4fab-87b1-877c1223f221`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:48:20,070::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:48:20,070::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6e1847ff-8627-42cf-a0da-67ea1c72240e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:48:20,071::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:48:20,071::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:48:20,071::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6e1847ff-8627-42cf-a0da-67ea1c72240e`::Granted request >Thread-338::DEBUG::2014-05-13 12:48:20,071::task::827::TaskManager.Task::(resourceAcquired) Task=`97492d73-2aac-4fab-87b1-877c1223f221`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:48:20,072::task::990::TaskManager.Task::(_decref) Task=`97492d73-2aac-4fab-87b1-877c1223f221`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:48:20,073::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:48:20,073::task::1185::TaskManager.Task::(prepare) Task=`97492d73-2aac-4fab-87b1-877c1223f221`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923395420160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:48:20,073::task::595::TaskManager.Task::(_updateState) Task=`97492d73-2aac-4fab-87b1-877c1223f221`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:48:20,073::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:48:20,073::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:48:20,074::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:48:20,074::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:48:20,074::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:48:20,074::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:48:20,074::task::990::TaskManager.Task::(_decref) Task=`97492d73-2aac-4fab-87b1-877c1223f221`::ref 0 aborting False >Thread-357::DEBUG::2014-05-13 12:48:20,808::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:48:20,816::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000434002 s, 901 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:48:20,897::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:48:20,988::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0476966 s, 21.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:48:21,103::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-338::DEBUG::2014-05-13 12:48:22,013::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2c8697c3] >Thread-338::DEBUG::2014-05-13 12:48:22,053::task::595::TaskManager.Task::(_updateState) Task=`948a9d1a-e2fe-464e-a3ee-788eba40a354`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:48:22,053::logUtils::44::dispatcher::(wrapper) Run and protect: createVolume(sdUUID='572b0496-5eab-41cd-a976-4514f111898e', spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', imgUUID='ac82bfa3-8cd5-4643-b36a-5ad28040fe0a', size='5368709120', volFormat=5, preallocate=2, diskType=2, volUUID='b4c44053-1991-4c45-aea6-2bcb369e4b65', desc='', srcImgUUID='00000000-0000-0000-0000-000000000000', srcVolUUID='00000000-0000-0000-0000-000000000000') >Thread-338::DEBUG::2014-05-13 12:48:22,054::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.572b0496-5eab-41cd-a976-4514f111898e`ReqID=`943ae171-276a-419e-ab7f-b5f61109eaeb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1464' at 'createVolume' >Thread-338::DEBUG::2014-05-13 12:48:22,054::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:48:22,054::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:48:22,054::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.572b0496-5eab-41cd-a976-4514f111898e`ReqID=`943ae171-276a-419e-ab7f-b5f61109eaeb`::Granted request >Thread-338::DEBUG::2014-05-13 12:48:22,054::task::827::TaskManager.Task::(resourceAcquired) Task=`948a9d1a-e2fe-464e-a3ee-788eba40a354`::_resourcesAcquired: Storage.572b0496-5eab-41cd-a976-4514f111898e (shared) >Thread-338::DEBUG::2014-05-13 12:48:22,055::task::990::TaskManager.Task::(_decref) Task=`948a9d1a-e2fe-464e-a3ee-788eba40a354`::ref 1 aborting False >Thread-338::DEBUG::2014-05-13 12:48:22,056::task::752::TaskManager.Task::(_save) Task=`948a9d1a-e2fe-464e-a3ee-788eba40a354`::_save: orig /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/948a9d1a-e2fe-464e-a3ee-788eba40a354 temp /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/948a9d1a-e2fe-464e-a3ee-788eba40a354.temp >Thread-338::DEBUG::2014-05-13 12:48:22,078::taskManager::68::TaskManager::(scheduleJob) scheduled job createVolume for task 948a9d1a-e2fe-464e-a3ee-788eba40a354 >Thread-338::INFO::2014-05-13 12:48:22,078::logUtils::47::dispatcher::(wrapper) Run and protect: createVolume, Return response: None >Thread-338::DEBUG::2014-05-13 12:48:22,078::task::1180::TaskManager.Task::(prepare) Task=`948a9d1a-e2fe-464e-a3ee-788eba40a354`::Prepare: 1 jobs exist, move to acquiring >Thread-338::DEBUG::2014-05-13 12:48:22,078::task::595::TaskManager.Task::(_updateState) Task=`948a9d1a-e2fe-464e-a3ee-788eba40a354`::moving from state preparing -> state acquiring >Thread-338::DEBUG::2014-05-13 12:48:22,079::task::752::TaskManager.Task::(_save) Task=`948a9d1a-e2fe-464e-a3ee-788eba40a354`::_save: orig /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/948a9d1a-e2fe-464e-a3ee-788eba40a354 temp /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/948a9d1a-e2fe-464e-a3ee-788eba40a354.temp >Thread-338::DEBUG::2014-05-13 12:48:22,087::task::595::TaskManager.Task::(_updateState) Task=`948a9d1a-e2fe-464e-a3ee-788eba40a354`::moving from state acquiring -> state queued >Thread-338::DEBUG::2014-05-13 12:48:22,087::task::752::TaskManager.Task::(_save) Task=`948a9d1a-e2fe-464e-a3ee-788eba40a354`::_save: orig /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/948a9d1a-e2fe-464e-a3ee-788eba40a354 temp /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/948a9d1a-e2fe-464e-a3ee-788eba40a354.temp >Thread-338::DEBUG::2014-05-13 12:48:22,094::taskManager::50::TaskManager::(_queueTask) queuing task: 948a9d1a-e2fe-464e-a3ee-788eba40a354 >Thread-338::DEBUG::2014-05-13 12:48:22,094::taskManager::56::TaskManager::(_queueTask) task queued: 948a9d1a-e2fe-464e-a3ee-788eba40a354 >0d92060a-0634-4295-965f-8c4fff978afc::DEBUG::2014-05-13 12:48:22,094::threadPool::57::Misc.ThreadPool::(setRunningTask) Number of running tasks: 1 >Thread-338::DEBUG::2014-05-13 12:48:22,094::task::1182::TaskManager.Task::(prepare) Task=`948a9d1a-e2fe-464e-a3ee-788eba40a354`::returning >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,094::threadPool::200::Misc.ThreadPool.WorkerThread::(_processNextTask) Task: 948a9d1a-e2fe-464e-a3ee-788eba40a354 running: <bound method Task.commit of <storage.task.Task instance at 0x7fa3cc0994d0>> with: None >Thread-338::DEBUG::2014-05-13 12:48:22,095::task::990::TaskManager.Task::(_decref) Task=`948a9d1a-e2fe-464e-a3ee-788eba40a354`::ref 0 aborting False >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,095::task::1194::TaskManager.Task::(commit) Task=`948a9d1a-e2fe-464e-a3ee-788eba40a354`::committing task: 948a9d1a-e2fe-464e-a3ee-788eba40a354 >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,095::task::595::TaskManager.Task::(_updateState) Task=`948a9d1a-e2fe-464e-a3ee-788eba40a354`::moving from state queued -> state running >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,096::task::752::TaskManager.Task::(_save) Task=`948a9d1a-e2fe-464e-a3ee-788eba40a354`::_save: orig /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/948a9d1a-e2fe-464e-a3ee-788eba40a354 temp /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/948a9d1a-e2fe-464e-a3ee-788eba40a354.temp >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,103::task::905::TaskManager.Task::(_runJobs) Task=`948a9d1a-e2fe-464e-a3ee-788eba40a354`::Task.run: running job 0: createVolume: <bound method StoragePool.createVolume of <storage.sp.StoragePool object at 0x7fa3bc680ed0>> (args: ('572b0496-5eab-41cd-a976-4514f111898e', 'ac82bfa3-8cd5-4643-b36a-5ad28040fe0a', 10485760, 5, 2, 2, 'b4c44053-1991-4c45-aea6-2bcb369e4b65', '', '00000000-0000-0000-0000-000000000000', '00000000-0000-0000-0000-000000000000') kwargs: {}) >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,103::task::331::TaskManager.Task::(run) Task=`948a9d1a-e2fe-464e-a3ee-788eba40a354`::Job.run: running createVolume: <bound method StoragePool.createVolume of <storage.sp.StoragePool object at 0x7fa3bc680ed0>> (args: ('572b0496-5eab-41cd-a976-4514f111898e', 'ac82bfa3-8cd5-4643-b36a-5ad28040fe0a', 10485760, 5, 2, 2, 'b4c44053-1991-4c45-aea6-2bcb369e4b65', '', '00000000-0000-0000-0000-000000000000', '00000000-0000-0000-0000-000000000000') kwargs: {}) callback None >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,103::resourceManager::198::ResourceManager.Request::(__init__) ResName=`572b0496-5eab-41cd-a976-4514f111898e_imageNS.ac82bfa3-8cd5-4643-b36a-5ad28040fe0a`ReqID=`f1360a9b-f80e-4c09-b940-10f231744585`::Request was made in '/usr/share/vdsm/storage/sp.py' line '1767' at 'createVolume' >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,104::resourceManager::542::ResourceManager::(registerResource) Trying to register resource '572b0496-5eab-41cd-a976-4514f111898e_imageNS.ac82bfa3-8cd5-4643-b36a-5ad28040fe0a' for lock type 'exclusive' >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,105::resourceFactories::125::Storage.ResourcesFactories::(__getResourceCandidatesList) Image ac82bfa3-8cd5-4643-b36a-5ad28040fe0a does not exist in domain 572b0496-5eab-41cd-a976-4514f111898e >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,105::resourceManager::601::ResourceManager::(registerResource) Resource '572b0496-5eab-41cd-a976-4514f111898e_imageNS.ac82bfa3-8cd5-4643-b36a-5ad28040fe0a' is free. Now locking as 'exclusive' (1 active user) >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,105::resourceManager::238::ResourceManager.Request::(grant) ResName=`572b0496-5eab-41cd-a976-4514f111898e_imageNS.ac82bfa3-8cd5-4643-b36a-5ad28040fe0a`ReqID=`f1360a9b-f80e-4c09-b940-10f231744585`::Granted request >948a9d1a-e2fe-464e-a3ee-788eba40a354::INFO::2014-05-13 12:48:22,106::image::123::Storage.Image::(create) Create placeholder /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/572b0496-5eab-41cd-a976-4514f111898e/images/ac82bfa3-8cd5-4643-b36a-5ad28040fe0a for image's volumes >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,106::task::752::TaskManager.Task::(_save) Task=`948a9d1a-e2fe-464e-a3ee-788eba40a354`::_save: orig /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/948a9d1a-e2fe-464e-a3ee-788eba40a354 temp /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/948a9d1a-e2fe-464e-a3ee-788eba40a354.temp >948a9d1a-e2fe-464e-a3ee-788eba40a354::INFO::2014-05-13 12:48:22,115::volume::443::Storage.Volume::(create) Creating volume b4c44053-1991-4c45-aea6-2bcb369e4b65 >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,115::task::752::TaskManager.Task::(_save) Task=`948a9d1a-e2fe-464e-a3ee-788eba40a354`::_save: orig /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/948a9d1a-e2fe-464e-a3ee-788eba40a354 temp /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/948a9d1a-e2fe-464e-a3ee-788eba40a354.temp >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,122::task::752::TaskManager.Task::(_save) Task=`948a9d1a-e2fe-464e-a3ee-788eba40a354`::_save: orig /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/948a9d1a-e2fe-464e-a3ee-788eba40a354 temp /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/948a9d1a-e2fe-464e-a3ee-788eba40a354.temp >948a9d1a-e2fe-464e-a3ee-788eba40a354::INFO::2014-05-13 12:48:22,132::fileVolume::144::Storage.Volume::(_create) Request to create RAW volume /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/572b0496-5eab-41cd-a976-4514f111898e/images/ac82bfa3-8cd5-4643-b36a-5ad28040fe0a/b4c44053-1991-4c45-aea6-2bcb369e4b65 with size = 10485760 sectors >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,135::task::752::TaskManager.Task::(_save) Task=`948a9d1a-e2fe-464e-a3ee-788eba40a354`::_save: orig /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/948a9d1a-e2fe-464e-a3ee-788eba40a354 temp /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/948a9d1a-e2fe-464e-a3ee-788eba40a354.temp >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,147::fileVolume::395::Storage.Volume::(newVolumeLease) Initializing volume lease volUUID=b4c44053-1991-4c45-aea6-2bcb369e4b65 sdUUID=572b0496-5eab-41cd-a976-4514f111898e, metaId=('/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/572b0496-5eab-41cd-a976-4514f111898e/images/ac82bfa3-8cd5-4643-b36a-5ad28040fe0a/b4c44053-1991-4c45-aea6-2bcb369e4b65',) >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,167::task::752::TaskManager.Task::(_save) Task=`948a9d1a-e2fe-464e-a3ee-788eba40a354`::_save: orig /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/948a9d1a-e2fe-464e-a3ee-788eba40a354 temp /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/948a9d1a-e2fe-464e-a3ee-788eba40a354.temp >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,175::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource '572b0496-5eab-41cd-a976-4514f111898e_imageNS.ac82bfa3-8cd5-4643-b36a-5ad28040fe0a' >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,175::resourceManager::635::ResourceManager::(releaseResource) Released resource '572b0496-5eab-41cd-a976-4514f111898e_imageNS.ac82bfa3-8cd5-4643-b36a-5ad28040fe0a' (0 active users) >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,175::resourceManager::641::ResourceManager::(releaseResource) Resource '572b0496-5eab-41cd-a976-4514f111898e_imageNS.ac82bfa3-8cd5-4643-b36a-5ad28040fe0a' is free, finding out if anyone is waiting for it. >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,175::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource '572b0496-5eab-41cd-a976-4514f111898e_imageNS.ac82bfa3-8cd5-4643-b36a-5ad28040fe0a', Clearing records. >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,175::task::595::TaskManager.Task::(_updateState) Task=`948a9d1a-e2fe-464e-a3ee-788eba40a354`::moving from state running -> state finished >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,176::task::752::TaskManager.Task::(_save) Task=`948a9d1a-e2fe-464e-a3ee-788eba40a354`::_save: orig /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/948a9d1a-e2fe-464e-a3ee-788eba40a354 temp /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/948a9d1a-e2fe-464e-a3ee-788eba40a354.temp >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,183::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.572b0496-5eab-41cd-a976-4514f111898e': < ResourceRef 'Storage.572b0496-5eab-41cd-a976-4514f111898e', isValid: 'True' obj: 'None'>} >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,184::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,184::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,184::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' (0 active users) >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,184::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' is free, finding out if anyone is waiting for it. >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,184::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e', Clearing records. >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,184::task::913::TaskManager.Task::(_runJobs) Task=`948a9d1a-e2fe-464e-a3ee-788eba40a354`::Task.run: exit - success: result {'uuid': 'b4c44053-1991-4c45-aea6-2bcb369e4b65'} >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,185::task::990::TaskManager.Task::(_decref) Task=`948a9d1a-e2fe-464e-a3ee-788eba40a354`::ref 0 aborting False >948a9d1a-e2fe-464e-a3ee-788eba40a354::DEBUG::2014-05-13 12:48:22,185::threadPool::57::Misc.ThreadPool::(setRunningTask) Number of running tasks: 0 >Thread-344::DEBUG::2014-05-13 12:48:22,411::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:48:22,420::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000398077 s, 874 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:48:23,001::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:48:23,083::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0386716 s, 26.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:48:24,121::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:48:25,096::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:48:25,170::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031431 s, 32.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:48:26,931::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:48:26,939::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000692573 s, 5.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:48:27,139::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:48:27,183::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:48:27,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0389779 s, 26.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:48:29,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:48:29,360::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0380843 s, 26.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:48:29,700::task::595::TaskManager.Task::(_updateState) Task=`abb4ecc7-a6ce-43bc-a0f0-0da0244c6019`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:48:29,700::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:48:29,701::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000434002', 'lastCheck': '8.9', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000692573', 'lastCheck': '2.8', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000398077', 'lastCheck': '7.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:48:29,701::task::1185::TaskManager.Task::(prepare) Task=`abb4ecc7-a6ce-43bc-a0f0-0da0244c6019`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000434002', 'lastCheck': '8.9', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000692573', 'lastCheck': '2.8', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000398077', 'lastCheck': '7.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:48:29,701::task::595::TaskManager.Task::(_updateState) Task=`abb4ecc7-a6ce-43bc-a0f0-0da0244c6019`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:48:29,701::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:48:29,701::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:48:29,702::task::990::TaskManager.Task::(_decref) Task=`abb4ecc7-a6ce-43bc-a0f0-0da0244c6019`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:48:30,109::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:48:30,109::task::595::TaskManager.Task::(_updateState) Task=`70b6dcda-abcc-4ea7-ae6d-20735eb046a4`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:48:30,109::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:48:30,110::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:48:30,110::task::1185::TaskManager.Task::(prepare) Task=`70b6dcda-abcc-4ea7-ae6d-20735eb046a4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:48:30,110::task::595::TaskManager.Task::(_updateState) Task=`70b6dcda-abcc-4ea7-ae6d-20735eb046a4`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:48:30,110::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:48:30,110::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:48:30,110::task::990::TaskManager.Task::(_decref) Task=`70b6dcda-abcc-4ea7-ae6d-20735eb046a4`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:48:30,116::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:48:30,116::task::595::TaskManager.Task::(_updateState) Task=`5025ece1-cdbe-48d5-a6c4-1463c99a9c2e`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:48:30,117::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:48:30,117::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`59d8f879-2389-4f53-a899-24b9792a88c5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:48:30,117::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:48:30,117::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:48:30,118::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`59d8f879-2389-4f53-a899-24b9792a88c5`::Granted request >Thread-338::DEBUG::2014-05-13 12:48:30,118::task::827::TaskManager.Task::(resourceAcquired) Task=`5025ece1-cdbe-48d5-a6c4-1463c99a9c2e`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:48:30,118::task::990::TaskManager.Task::(_decref) Task=`5025ece1-cdbe-48d5-a6c4-1463c99a9c2e`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:48:30,119::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923394306048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:48:30,119::task::1185::TaskManager.Task::(prepare) Task=`5025ece1-cdbe-48d5-a6c4-1463c99a9c2e`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923394306048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:48:30,119::task::595::TaskManager.Task::(_updateState) Task=`5025ece1-cdbe-48d5-a6c4-1463c99a9c2e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:48:30,120::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:48:30,120::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:48:30,120::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:48:30,120::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:48:30,120::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:48:30,121::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:48:30,121::task::990::TaskManager.Task::(_decref) Task=`5025ece1-cdbe-48d5-a6c4-1463c99a9c2e`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:48:30,158::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-357::DEBUG::2014-05-13 12:48:30,827::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:48:30,835::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000404789 s, 966 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:48:31,373::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:48:31,454::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0395282 s, 25.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:48:32,057::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:48:32,057::task::595::TaskManager.Task::(_updateState) Task=`94c7da46-e7da-425e-a9cb-31fd5bf61342`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:48:32,058::logUtils::44::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-66::DEBUG::2014-05-13 12:48:32,058::taskManager::102::TaskManager::(getAllTasksStatuses) Entry. >Thread-66::DEBUG::2014-05-13 12:48:32,058::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 948a9d1a-e2fe-464e-a3ee-788eba40a354 >Thread-66::DEBUG::2014-05-13 12:48:32,058::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '948a9d1a-e2fe-464e-a3ee-788eba40a354'} >Thread-66::DEBUG::2014-05-13 12:48:32,058::taskManager::113::TaskManager::(getAllTasksStatuses) Return: {'948a9d1a-e2fe-464e-a3ee-788eba40a354': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '948a9d1a-e2fe-464e-a3ee-788eba40a354'}} >Thread-66::INFO::2014-05-13 12:48:32,058::logUtils::47::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {'948a9d1a-e2fe-464e-a3ee-788eba40a354': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '948a9d1a-e2fe-464e-a3ee-788eba40a354'}}} >Thread-66::DEBUG::2014-05-13 12:48:32,059::task::1185::TaskManager.Task::(prepare) Task=`94c7da46-e7da-425e-a9cb-31fd5bf61342`::finished: {'allTasksStatus': {'948a9d1a-e2fe-464e-a3ee-788eba40a354': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '948a9d1a-e2fe-464e-a3ee-788eba40a354'}}} >Thread-66::DEBUG::2014-05-13 12:48:32,059::task::595::TaskManager.Task::(_updateState) Task=`94c7da46-e7da-425e-a9cb-31fd5bf61342`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:48:32,059::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:48:32,059::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:48:32,059::task::990::TaskManager.Task::(_decref) Task=`94c7da46-e7da-425e-a9cb-31fd5bf61342`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:48:32,093::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [2c8697c3] >Thread-338::DEBUG::2014-05-13 12:48:32,094::task::595::TaskManager.Task::(_updateState) Task=`0f390528-9713-4cb8-ab71-1284deb40ebe`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:48:32,094::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeInfo(sdUUID='572b0496-5eab-41cd-a976-4514f111898e', spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', imgUUID='ac82bfa3-8cd5-4643-b36a-5ad28040fe0a', volUUID='b4c44053-1991-4c45-aea6-2bcb369e4b65', options=None) >Thread-338::DEBUG::2014-05-13 12:48:32,094::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.572b0496-5eab-41cd-a976-4514f111898e`ReqID=`d3263ae5-1364-4c4a-9f4b-42db1d581675`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '3089' at 'getVolumeInfo' >Thread-338::DEBUG::2014-05-13 12:48:32,094::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:48:32,095::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:48:32,095::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.572b0496-5eab-41cd-a976-4514f111898e`ReqID=`d3263ae5-1364-4c4a-9f4b-42db1d581675`::Granted request >Thread-338::DEBUG::2014-05-13 12:48:32,095::task::827::TaskManager.Task::(resourceAcquired) Task=`0f390528-9713-4cb8-ab71-1284deb40ebe`::_resourcesAcquired: Storage.572b0496-5eab-41cd-a976-4514f111898e (shared) >Thread-338::DEBUG::2014-05-13 12:48:32,095::task::990::TaskManager.Task::(_decref) Task=`0f390528-9713-4cb8-ab71-1284deb40ebe`::ref 1 aborting False >Thread-338::DEBUG::2014-05-13 12:48:32,097::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for b4c44053-1991-4c45-aea6-2bcb369e4b65 >Thread-338::INFO::2014-05-13 12:48:32,099::volume::879::Storage.Volume::(getInfo) Info request: sdUUID=572b0496-5eab-41cd-a976-4514f111898e imgUUID=ac82bfa3-8cd5-4643-b36a-5ad28040fe0a volUUID = b4c44053-1991-4c45-aea6-2bcb369e4b65 >Thread-338::INFO::2014-05-13 12:48:32,125::volume::909::Storage.Volume::(getInfo) 572b0496-5eab-41cd-a976-4514f111898e/ac82bfa3-8cd5-4643-b36a-5ad28040fe0a/b4c44053-1991-4c45-aea6-2bcb369e4b65 info is {'status': 'OK', 'domain': '572b0496-5eab-41cd-a976-4514f111898e', 'voltype': 'LEAF', 'description': '', 'parent': '00000000-0000-0000-0000-000000000000', 'format': 'RAW', 'image': 'ac82bfa3-8cd5-4643-b36a-5ad28040fe0a', 'ctime': '1399974502', 'disktype': '2', 'legality': 'LEGAL', 'mtime': '1399974502', 'apparentsize': '5368709120', 'children': [], 'pool': '', 'capacity': '5368709120', 'uuid': 'b4c44053-1991-4c45-aea6-2bcb369e4b65', 'truesize': '24576', 'type': 'SPARSE'} >Thread-338::INFO::2014-05-13 12:48:32,125::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeInfo, Return response: {'info': {'status': 'OK', 'domain': '572b0496-5eab-41cd-a976-4514f111898e', 'voltype': 'LEAF', 'description': '', 'parent': '00000000-0000-0000-0000-000000000000', 'format': 'RAW', 'image': 'ac82bfa3-8cd5-4643-b36a-5ad28040fe0a', 'ctime': '1399974502', 'disktype': '2', 'legality': 'LEGAL', 'mtime': '1399974502', 'apparentsize': '5368709120', 'children': [], 'pool': '', 'capacity': '5368709120', 'uuid': 'b4c44053-1991-4c45-aea6-2bcb369e4b65', 'truesize': '24576', 'type': 'SPARSE'}} >Thread-338::DEBUG::2014-05-13 12:48:32,125::task::1185::TaskManager.Task::(prepare) Task=`0f390528-9713-4cb8-ab71-1284deb40ebe`::finished: {'info': {'status': 'OK', 'domain': '572b0496-5eab-41cd-a976-4514f111898e', 'voltype': 'LEAF', 'description': '', 'parent': '00000000-0000-0000-0000-000000000000', 'format': 'RAW', 'image': 'ac82bfa3-8cd5-4643-b36a-5ad28040fe0a', 'ctime': '1399974502', 'disktype': '2', 'legality': 'LEGAL', 'mtime': '1399974502', 'apparentsize': '5368709120', 'children': [], 'pool': '', 'capacity': '5368709120', 'uuid': 'b4c44053-1991-4c45-aea6-2bcb369e4b65', 'truesize': '24576', 'type': 'SPARSE'}} >Thread-338::DEBUG::2014-05-13 12:48:32,125::task::595::TaskManager.Task::(_updateState) Task=`0f390528-9713-4cb8-ab71-1284deb40ebe`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:48:32,126::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.572b0496-5eab-41cd-a976-4514f111898e': < ResourceRef 'Storage.572b0496-5eab-41cd-a976-4514f111898e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:48:32,126::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:48:32,126::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' >Thread-338::DEBUG::2014-05-13 12:48:32,126::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:48:32,126::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:48:32,127::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:48:32,127::task::990::TaskManager.Task::(_decref) Task=`0f390528-9713-4cb8-ab71-1284deb40ebe`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:48:32,161::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:48:32,166::task::595::TaskManager.Task::(_updateState) Task=`25dc9c10-96ce-400a-b460-11166d1c9f03`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:48:32,167::logUtils::44::dispatcher::(wrapper) Run and protect: clearTask(taskID='948a9d1a-e2fe-464e-a3ee-788eba40a354', spUUID=None, options=None) >Thread-66::DEBUG::2014-05-13 12:48:32,167::taskManager::161::TaskManager::(clearTask) Entry. taskID: 948a9d1a-e2fe-464e-a3ee-788eba40a354 >Thread-66::DEBUG::2014-05-13 12:48:32,168::taskManager::166::TaskManager::(clearTask) Return. >Thread-66::INFO::2014-05-13 12:48:32,168::logUtils::47::dispatcher::(wrapper) Run and protect: clearTask, Return response: None >Thread-66::DEBUG::2014-05-13 12:48:32,168::task::1185::TaskManager.Task::(prepare) Task=`25dc9c10-96ce-400a-b460-11166d1c9f03`::finished: None >Thread-66::DEBUG::2014-05-13 12:48:32,168::task::595::TaskManager.Task::(_updateState) Task=`25dc9c10-96ce-400a-b460-11166d1c9f03`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:48:32,168::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:48:32,169::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:48:32,169::task::990::TaskManager.Task::(_decref) Task=`25dc9c10-96ce-400a-b460-11166d1c9f03`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:48:32,431::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:48:32,439::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000443978 s, 784 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:48:33,179::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:48:33,467::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:48:33,876::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.366744 s, 2.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:48:35,889::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:48:35,971::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0393293 s, 26.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:48:36,197::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:48:36,946::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:48:36,955::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000842825 s, 4.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:48:37,984::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:48:38,067::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0390408 s, 26.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:48:39,215::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:48:39,373::BindingXMLRPC::1067::vds::(wrapper) client [10.35.163.66]::call vmCreate with ({'acpiEnable': 'true', 'emulatedMachine': 'rhel6.5.0', 'vmId': '4da6d3ed-e2fd-4318-a58f-715cc1756430', 'memGuaranteedSize': 1024, 'spiceSslCipherSuite': 'DEFAULT', 'timeOffset': '0', 'cpuType': 'Nehalem', 'custom': {}, 'smp': '1', 'vmType': 'kvm', 'memSize': 1024, 'smpCoresPerSocket': '1', 'vmName': 'vm1', 'nice': '0', 'smartcardEnable': 'false', 'keyboardLayout': 'en-us', 'kvmEnable': 'true', 'transparentHugePages': 'true', 'displayNetwork': 'rhevm', 'devices': [{'device': 'qxl', 'specParams': {'vram': '32768', 'heads': '1'}, 'type': 'video', 'deviceId': 'c069643e-9153-4dfd-9e0a-25edd2633e21'}, {'index': '2', 'iface': 'ide', 'specParams': {'path': ''}, 'readonly': 'true', 'deviceId': 'd98061e3-2535-4744-acc2-94749b17ebee', 'path': '', 'device': 'cdrom', 'shared': 'false', 'type': 'disk'}, {'index': 0, 'iface': 'scsi', 'format': 'raw', 'optional': 'false', 'poolID': 'aa069361-8e06-4459-a17b-748c0a2a828e', 'volumeID': 'b4c44053-1991-4c45-aea6-2bcb369e4b65', 'imageID': 'ac82bfa3-8cd5-4643-b36a-5ad28040fe0a', 'specParams': {}, 'readonly': 'false', 'domainID': '572b0496-5eab-41cd-a976-4514f111898e', 'deviceId': 'ac82bfa3-8cd5-4643-b36a-5ad28040fe0a', 'address': {'bus': '0', 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'}, 'device': 'disk', 'shared': 'false', 'propagateErrors': 'off', 'type': 'disk'}, {'device': 'memballoon', 'specParams': {'model': 'virtio'}, 'type': 'balloon', 'deviceId': 'a7b4d724-69cb-4853-9d96-453972b787a2'}, {'index': '0', 'specParams': {}, 'deviceId': '248e16a5-83c2-4a93-acc5-5af785d9e9d3', 'device': 'scsi', 'model': 'virtio-scsi', 'type': 'controller'}], 'maxVCpus': '160', 'spiceSecureChannels': 'smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard', 'display': 'qxl'},) {} flowID [51215822] >Thread-66::INFO::2014-05-13 12:48:39,380::clientIF::366::vds::(createVm) vmContainerLock acquired by vm 4da6d3ed-e2fd-4318-a58f-715cc1756430 >Thread-66::DEBUG::2014-05-13 12:48:39,381::libvirtconnection::145::root::(get) trying to connect libvirt >Thread-66::DEBUG::2014-05-13 12:48:39,420::clientIF::379::vds::(createVm) Total desktops after creation of 4da6d3ed-e2fd-4318-a58f-715cc1756430 is 1 >Thread-365::DEBUG::2014-05-13 12:48:39,420::vm::2238::vm.Vm::(_startUnderlyingVm) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Start >Thread-66::DEBUG::2014-05-13 12:48:39,421::BindingXMLRPC::1074::vds::(wrapper) return vmCreate with {'status': {'message': 'Done', 'code': 0}, 'vmList': {'status': 'WaitForLaunch', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.5.0', 'vmId': '4da6d3ed-e2fd-4318-a58f-715cc1756430', 'pid': '0', 'memGuaranteedSize': 1024, 'timeOffset': '0', 'keyboardLayout': 'en-us', 'displayPort': '-1', 'displaySecurePort': '-1', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Nehalem', 'smp': '1', 'clientIp': '', 'nicModel': 'rtl8139,pv', 'smartcardEnable': 'false', 'kvmEnable': 'true', 'transparentHugePages': 'true', 'displayNetwork': 'rhevm', 'devices': [{'device': 'qxl', 'specParams': {'vram': '32768', 'heads': '1'}, 'type': 'video', 'deviceId': 'c069643e-9153-4dfd-9e0a-25edd2633e21'}, {'index': '2', 'iface': 'ide', 'specParams': {'path': ''}, 'readonly': 'true', 'deviceId': 'd98061e3-2535-4744-acc2-94749b17ebee', 'path': '', 'device': 'cdrom', 'shared': 'false', 'type': 'disk'}, {'index': 0, 'iface': 'scsi', 'format': 'raw', 'optional': 'false', 'poolID': 'aa069361-8e06-4459-a17b-748c0a2a828e', 'volumeID': 'b4c44053-1991-4c45-aea6-2bcb369e4b65', 'imageID': 'ac82bfa3-8cd5-4643-b36a-5ad28040fe0a', 'specParams': {}, 'readonly': 'false', 'domainID': '572b0496-5eab-41cd-a976-4514f111898e', 'deviceId': 'ac82bfa3-8cd5-4643-b36a-5ad28040fe0a', 'address': {'bus': '0', 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'}, 'device': 'disk', 'shared': 'false', 'propagateErrors': 'off', 'type': 'disk'}, {'device': 'memballoon', 'specParams': {'model': 'virtio'}, 'type': 'balloon', 'deviceId': 'a7b4d724-69cb-4853-9d96-453972b787a2'}, {'index': '0', 'specParams': {}, 'deviceId': '248e16a5-83c2-4a93-acc5-5af785d9e9d3', 'device': 'scsi', 'model': 'virtio-scsi', 'type': 'controller'}], 'custom': {}, 'vmType': 'kvm', 'memSize': 1024, 'displayIp': '10.35.116.3', 'maxVCpus': '160', 'spiceSecureChannels': 'smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard', 'smpCoresPerSocket': '1', 'vmName': 'vm1', 'display': 'qxl', 'nice': '0'}} >Thread-365::DEBUG::2014-05-13 12:48:39,422::vm::2242::vm.Vm::(_startUnderlyingVm) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::_ongoingCreations acquired >Thread-365::INFO::2014-05-13 12:48:39,423::vm::3112::vm.Vm::(_run) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::VM wrapper has started >Thread-365::DEBUG::2014-05-13 12:48:39,425::task::595::TaskManager.Task::(_updateState) Task=`40793fc8-2165-490b-9cb8-e5a5189ae17e`::moving from state init -> state preparing >Thread-365::INFO::2014-05-13 12:48:39,426::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='572b0496-5eab-41cd-a976-4514f111898e', spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', imgUUID='ac82bfa3-8cd5-4643-b36a-5ad28040fe0a', volUUID='b4c44053-1991-4c45-aea6-2bcb369e4b65', options=None) >Thread-365::INFO::2014-05-13 12:48:39,431::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '24576', 'apparentsize': '5368709120'} >Thread-365::DEBUG::2014-05-13 12:48:39,431::task::1185::TaskManager.Task::(prepare) Task=`40793fc8-2165-490b-9cb8-e5a5189ae17e`::finished: {'truesize': '24576', 'apparentsize': '5368709120'} >Thread-365::DEBUG::2014-05-13 12:48:39,432::task::595::TaskManager.Task::(_updateState) Task=`40793fc8-2165-490b-9cb8-e5a5189ae17e`::moving from state preparing -> state finished >Thread-365::DEBUG::2014-05-13 12:48:39,432::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-365::DEBUG::2014-05-13 12:48:39,432::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-365::DEBUG::2014-05-13 12:48:39,432::task::990::TaskManager.Task::(_decref) Task=`40793fc8-2165-490b-9cb8-e5a5189ae17e`::ref 0 aborting False >Thread-365::INFO::2014-05-13 12:48:39,432::clientIF::324::vds::(prepareVolumePath) prepared volume path: >Thread-365::DEBUG::2014-05-13 12:48:39,433::task::595::TaskManager.Task::(_updateState) Task=`d8bbb4d5-50ca-48f3-aee4-e86f043f4052`::moving from state init -> state preparing >Thread-365::INFO::2014-05-13 12:48:39,433::logUtils::44::dispatcher::(wrapper) Run and protect: prepareImage(sdUUID='572b0496-5eab-41cd-a976-4514f111898e', spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', imgUUID='ac82bfa3-8cd5-4643-b36a-5ad28040fe0a', leafUUID='b4c44053-1991-4c45-aea6-2bcb369e4b65') >Thread-365::DEBUG::2014-05-13 12:48:39,433::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.572b0496-5eab-41cd-a976-4514f111898e`ReqID=`c1267dbe-720a-4c73-a72b-13e5fddb5c4b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '3248' at 'prepareImage' >Thread-365::DEBUG::2014-05-13 12:48:39,433::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' for lock type 'shared' >Thread-365::DEBUG::2014-05-13 12:48:39,434::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' is free. Now locking as 'shared' (1 active user) >Thread-365::DEBUG::2014-05-13 12:48:39,434::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.572b0496-5eab-41cd-a976-4514f111898e`ReqID=`c1267dbe-720a-4c73-a72b-13e5fddb5c4b`::Granted request >Thread-365::DEBUG::2014-05-13 12:48:39,434::task::827::TaskManager.Task::(resourceAcquired) Task=`d8bbb4d5-50ca-48f3-aee4-e86f043f4052`::_resourcesAcquired: Storage.572b0496-5eab-41cd-a976-4514f111898e (shared) >Thread-365::DEBUG::2014-05-13 12:48:39,434::task::990::TaskManager.Task::(_decref) Task=`d8bbb4d5-50ca-48f3-aee4-e86f043f4052`::ref 1 aborting False >Thread-365::DEBUG::2014-05-13 12:48:39,438::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for b4c44053-1991-4c45-aea6-2bcb369e4b65 >Thread-365::INFO::2014-05-13 12:48:39,440::logUtils::47::dispatcher::(wrapper) Run and protect: prepareImage, Return response: {'info': {'domainID': '572b0496-5eab-41cd-a976-4514f111898e', 'volType': 'path', 'leaseOffset': 0, 'path': '/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/images/ac82bfa3-8cd5-4643-b36a-5ad28040fe0a/b4c44053-1991-4c45-aea6-2bcb369e4b65', 'volumeID': 'b4c44053-1991-4c45-aea6-2bcb369e4b65', 'leasePath': '/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/images/ac82bfa3-8cd5-4643-b36a-5ad28040fe0a/b4c44053-1991-4c45-aea6-2bcb369e4b65.lease', 'imageID': 'ac82bfa3-8cd5-4643-b36a-5ad28040fe0a'}, 'path': '/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/572b0496-5eab-41cd-a976-4514f111898e/images/ac82bfa3-8cd5-4643-b36a-5ad28040fe0a/b4c44053-1991-4c45-aea6-2bcb369e4b65', 'imgVolumesInfo': [{'domainID': '572b0496-5eab-41cd-a976-4514f111898e', 'volType': 'path', 'leaseOffset': 0, 'path': '/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/images/ac82bfa3-8cd5-4643-b36a-5ad28040fe0a/b4c44053-1991-4c45-aea6-2bcb369e4b65', 'volumeID': 'b4c44053-1991-4c45-aea6-2bcb369e4b65', 'leasePath': '/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/images/ac82bfa3-8cd5-4643-b36a-5ad28040fe0a/b4c44053-1991-4c45-aea6-2bcb369e4b65.lease', 'imageID': 'ac82bfa3-8cd5-4643-b36a-5ad28040fe0a'}]} >Thread-365::DEBUG::2014-05-13 12:48:39,440::task::1185::TaskManager.Task::(prepare) Task=`d8bbb4d5-50ca-48f3-aee4-e86f043f4052`::finished: {'info': {'domainID': '572b0496-5eab-41cd-a976-4514f111898e', 'volType': 'path', 'leaseOffset': 0, 'path': '/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/images/ac82bfa3-8cd5-4643-b36a-5ad28040fe0a/b4c44053-1991-4c45-aea6-2bcb369e4b65', 'volumeID': 'b4c44053-1991-4c45-aea6-2bcb369e4b65', 'leasePath': '/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/images/ac82bfa3-8cd5-4643-b36a-5ad28040fe0a/b4c44053-1991-4c45-aea6-2bcb369e4b65.lease', 'imageID': 'ac82bfa3-8cd5-4643-b36a-5ad28040fe0a'}, 'path': '/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/572b0496-5eab-41cd-a976-4514f111898e/images/ac82bfa3-8cd5-4643-b36a-5ad28040fe0a/b4c44053-1991-4c45-aea6-2bcb369e4b65', 'imgVolumesInfo': [{'domainID': '572b0496-5eab-41cd-a976-4514f111898e', 'volType': 'path', 'leaseOffset': 0, 'path': '/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/images/ac82bfa3-8cd5-4643-b36a-5ad28040fe0a/b4c44053-1991-4c45-aea6-2bcb369e4b65', 'volumeID': 'b4c44053-1991-4c45-aea6-2bcb369e4b65', 'leasePath': '/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/images/ac82bfa3-8cd5-4643-b36a-5ad28040fe0a/b4c44053-1991-4c45-aea6-2bcb369e4b65.lease', 'imageID': 'ac82bfa3-8cd5-4643-b36a-5ad28040fe0a'}]} >Thread-365::DEBUG::2014-05-13 12:48:39,440::task::595::TaskManager.Task::(_updateState) Task=`d8bbb4d5-50ca-48f3-aee4-e86f043f4052`::moving from state preparing -> state finished >Thread-365::DEBUG::2014-05-13 12:48:39,440::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.572b0496-5eab-41cd-a976-4514f111898e': < ResourceRef 'Storage.572b0496-5eab-41cd-a976-4514f111898e', isValid: 'True' obj: 'None'>} >Thread-365::DEBUG::2014-05-13 12:48:39,440::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-365::DEBUG::2014-05-13 12:48:39,441::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' >Thread-365::DEBUG::2014-05-13 12:48:39,441::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' (0 active users) >Thread-365::DEBUG::2014-05-13 12:48:39,441::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' is free, finding out if anyone is waiting for it. >Thread-365::DEBUG::2014-05-13 12:48:39,441::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e', Clearing records. >Thread-365::DEBUG::2014-05-13 12:48:39,441::task::990::TaskManager.Task::(_decref) Task=`d8bbb4d5-50ca-48f3-aee4-e86f043f4052`::ref 0 aborting False >Thread-365::INFO::2014-05-13 12:48:39,441::clientIF::324::vds::(prepareVolumePath) prepared volume path: /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/572b0496-5eab-41cd-a976-4514f111898e/images/ac82bfa3-8cd5-4643-b36a-5ad28040fe0a/b4c44053-1991-4c45-aea6-2bcb369e4b65 >Thread-365::DEBUG::2014-05-13 12:48:39,453::vm::3151::vm.Vm::(_run) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::<?xml version="1.0" encoding="utf-8"?> ><domain type="kvm"> > <name>vm1</name> > <uuid>4da6d3ed-e2fd-4318-a58f-715cc1756430</uuid> > <memory>1048576</memory> > <currentMemory>1048576</currentMemory> > <vcpu current="1">160</vcpu> > <memtune> > <min_guarantee>1048576</min_guarantee> > </memtune> > <devices> > <channel type="unix"> > <target name="com.redhat.rhevm.vdsm" type="virtio"/> > <source mode="bind" path="/var/lib/libvirt/qemu/channels/4da6d3ed-e2fd-4318-a58f-715cc1756430.com.redhat.rhevm.vdsm"/> > </channel> > <channel type="unix"> > <target name="org.qemu.guest_agent.0" type="virtio"/> > <source mode="bind" path="/var/lib/libvirt/qemu/channels/4da6d3ed-e2fd-4318-a58f-715cc1756430.org.qemu.guest_agent.0"/> > </channel> > <input bus="ps2" type="mouse"/> > <channel type="spicevmc"> > <target name="com.redhat.spice.0" type="virtio"/> > </channel> > <graphics autoport="yes" keymap="en-us" passwd="*****" passwdValidTo="1970-01-01T00:00:01" port="-1" tlsPort="-1" type="spice"> > <channel mode="secure" name="main"/> > <channel mode="secure" name="inputs"/> > <channel mode="secure" name="cursor"/> > <channel mode="secure" name="playback"/> > <channel mode="secure" name="record"/> > <channel mode="secure" name="display"/> > <channel mode="secure" name="usbredir"/> > <channel mode="secure" name="smartcard"/> > <listen network="vdsm-rhevm" type="network"/> > </graphics> > <memballoon model="virtio"/> > <controller index="0" model="virtio-scsi" type="scsi"/> > <video> > <model heads="1" type="qxl" vram="32768"/> > </video> > <disk device="cdrom" snapshot="no" type="file"> > <source file="" startupPolicy="optional"/> > <target bus="ide" dev="hdc"/> > <readonly/> > <serial/> > </disk> > <disk device="disk" snapshot="no" type="file"> > <address bus="0" controller="0" target="0" type="drive" unit="0"/> > <source file="/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/572b0496-5eab-41cd-a976-4514f111898e/images/ac82bfa3-8cd5-4643-b36a-5ad28040fe0a/b4c44053-1991-4c45-aea6-2bcb369e4b65"/> > <target bus="scsi" dev="sda"/> > <serial>ac82bfa3-8cd5-4643-b36a-5ad28040fe0a</serial> > <driver cache="none" error_policy="stop" io="threads" name="qemu" type="raw"/> > </disk> > </devices> > <os> > <type arch="x86_64" machine="rhel6.5.0">hvm</type> > <smbios mode="sysinfo"/> > </os> > <sysinfo type="smbios"> > <system> > <entry name="manufacturer">Red Hat</entry> > <entry name="product">RHEV Hypervisor</entry> > <entry name="version">6Server-6.5.0.1.el6</entry> > <entry name="serial">13b1aa2c-2ad7-4ab9-afd9-fdab84f5026f</entry> > <entry name="uuid">4da6d3ed-e2fd-4318-a58f-715cc1756430</entry> > </system> > </sysinfo> > <clock adjustment="0" offset="variable"> > <timer name="rtc" tickpolicy="catchup"/> > </clock> > <features> > <acpi/> > </features> > <cpu match="exact"> > <model>Nehalem</model> > <topology cores="1" sockets="160" threads="1"/> > </cpu> ></domain> > >libvirtEventLoop::DEBUG::2014-05-13 12:48:39,980::vm::5151::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::event Started detail 0 opaque None >Thread-365::DEBUG::2014-05-13 12:48:40,003::vm::5007::vm.Vm::(_getUnderlyingDriveInfo) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Looking for drive with attributes {'name': u'hdc', 'bootOrder': '', 'boot': [], 'readonly': True, 'address': {u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, 'path': '', 'type': u'cdrom'} >Thread-365::DEBUG::2014-05-13 12:48:40,003::vm::5028::vm.Vm::(_getUnderlyingDriveInfo) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Matched {'name': (u'hdc', u'hdc'), 'bootOrder': ('', None), 'boot': ([], None), 'readonly': (True, True), 'address': ({u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, {u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}), 'path': ('', ''), 'type': (u'cdrom', u'cdrom')} >Thread-365::DEBUG::2014-05-13 12:48:40,004::vm::5047::vm.Vm::(_getUnderlyingDriveInfo) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Matched {'name': (u'hdc', u'hdc'), 'bootOrder': ('', None), 'boot': ([], None), 'readonly': (True, 'True'), 'address': ({u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, {u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}), 'path': ('', ''), 'type': (u'cdrom', 'disk')} >Thread-365::DEBUG::2014-05-13 12:48:40,004::vm::5007::vm.Vm::(_getUnderlyingDriveInfo) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Looking for drive with attributes {'name': u'sda', 'bootOrder': '', 'boot': [], 'readonly': False, 'address': {u'bus': u'0', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, 'path': u'/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/572b0496-5eab-41cd-a976-4514f111898e/images/ac82bfa3-8cd5-4643-b36a-5ad28040fe0a/b4c44053-1991-4c45-aea6-2bcb369e4b65', 'type': u'disk'} >Thread-365::DEBUG::2014-05-13 12:48:40,004::vm::5028::vm.Vm::(_getUnderlyingDriveInfo) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Matched {'name': (u'sda', u'sda'), 'bootOrder': ('', None), 'boot': ([], None), 'readonly': (False, False), 'address': ({u'bus': u'0', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, {u'bus': u'0', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}), 'path': (u'/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/572b0496-5eab-41cd-a976-4514f111898e/images/ac82bfa3-8cd5-4643-b36a-5ad28040fe0a/b4c44053-1991-4c45-aea6-2bcb369e4b65', '/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/572b0496-5eab-41cd-a976-4514f111898e/images/ac82bfa3-8cd5-4643-b36a-5ad28040fe0a/b4c44053-1991-4c45-aea6-2bcb369e4b65'), 'type': (u'disk', u'disk')} >Thread-365::DEBUG::2014-05-13 12:48:40,004::vm::5047::vm.Vm::(_getUnderlyingDriveInfo) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Matched {'name': (u'sda', u'sda'), 'bootOrder': ('', None), 'boot': ([], None), 'readonly': (False, 'False'), 'address': ({u'bus': u'0', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, {u'bus': u'0', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}), 'path': (u'/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/572b0496-5eab-41cd-a976-4514f111898e/images/ac82bfa3-8cd5-4643-b36a-5ad28040fe0a/b4c44053-1991-4c45-aea6-2bcb369e4b65', '/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/572b0496-5eab-41cd-a976-4514f111898e/images/ac82bfa3-8cd5-4643-b36a-5ad28040fe0a/b4c44053-1991-4c45-aea6-2bcb369e4b65'), 'type': (u'disk', 'disk')} >Thread-365::DEBUG::2014-05-13 12:48:40,045::sampling::285::vm.Vm::(start) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Start statistics collection >Thread-367::DEBUG::2014-05-13 12:48:40,045::sampling::314::vm.Vm::(run) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Stats thread started >Thread-367::DEBUG::2014-05-13 12:48:40,046::task::595::TaskManager.Task::(_updateState) Task=`4f2b03d7-f79e-4410-8f7a-15952097d053`::moving from state init -> state preparing >Thread-367::INFO::2014-05-13 12:48:40,046::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='572b0496-5eab-41cd-a976-4514f111898e', spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', imgUUID='ac82bfa3-8cd5-4643-b36a-5ad28040fe0a', volUUID='b4c44053-1991-4c45-aea6-2bcb369e4b65', options=None) >Thread-367::INFO::2014-05-13 12:48:40,049::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '24576', 'apparentsize': '5368709120'} >Thread-367::DEBUG::2014-05-13 12:48:40,049::task::1185::TaskManager.Task::(prepare) Task=`4f2b03d7-f79e-4410-8f7a-15952097d053`::finished: {'truesize': '24576', 'apparentsize': '5368709120'} >Thread-367::DEBUG::2014-05-13 12:48:40,049::task::595::TaskManager.Task::(_updateState) Task=`4f2b03d7-f79e-4410-8f7a-15952097d053`::moving from state preparing -> state finished >Thread-367::DEBUG::2014-05-13 12:48:40,049::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-367::DEBUG::2014-05-13 12:48:40,050::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-367::DEBUG::2014-05-13 12:48:40,050::task::990::TaskManager.Task::(_decref) Task=`4f2b03d7-f79e-4410-8f7a-15952097d053`::ref 0 aborting False >Thread-365::DEBUG::2014-05-13 12:48:40,058::vmChannels::194::vds::(register) Add fileno 185 to listener's channels. >Thread-365::WARNING::2014-05-13 12:48:40,060::vm::3676::vm.Vm::(_readPauseCode) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::_readPauseCode unsupported by libvirt vm >Dummy-337::DEBUG::2014-05-13 12:48:40,079::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-365::DEBUG::2014-05-13 12:48:40,153::vm::2259::vm.Vm::(_startUnderlyingVm) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::_ongoingCreations released >Thread-66::DEBUG::2014-05-13 12:48:40,170::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:48:40,173::task::595::TaskManager.Task::(_updateState) Task=`08ea09bc-06be-4827-a154-97ee79146933`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:48:40,176::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:48:40,201::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:48:40,202::task::1185::TaskManager.Task::(prepare) Task=`08ea09bc-06be-4827-a154-97ee79146933`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:48:40,204::task::595::TaskManager.Task::(_updateState) Task=`08ea09bc-06be-4827-a154-97ee79146933`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:48:40,206::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:48:40,208::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:48:40,210::task::990::TaskManager.Task::(_decref) Task=`08ea09bc-06be-4827-a154-97ee79146933`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:48:40,222::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:48:40,225::task::595::TaskManager.Task::(_updateState) Task=`3b3870f7-9c69-4b9c-b5df-f5d4ca2b3e91`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:48:40,228::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:48:40,232::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`431dc915-3e7e-4ba0-aa73-8eec0f44e710`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:48:40,233::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:48:40,235::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:48:40,236::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`431dc915-3e7e-4ba0-aa73-8eec0f44e710`::Granted request >Thread-338::DEBUG::2014-05-13 12:48:40,240::task::827::TaskManager.Task::(resourceAcquired) Task=`3b3870f7-9c69-4b9c-b5df-f5d4ca2b3e91`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:48:40,241::task::990::TaskManager.Task::(_decref) Task=`3b3870f7-9c69-4b9c-b5df-f5d4ca2b3e91`::ref 1 aborting False >Dummy-337::DEBUG::2014-05-13 12:48:40,244::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320567 s, 31.9 MB/s\n'; <rc> = 0 >Thread-338::INFO::2014-05-13 12:48:40,249::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923394306048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:48:40,250::task::1185::TaskManager.Task::(prepare) Task=`3b3870f7-9c69-4b9c-b5df-f5d4ca2b3e91`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923394306048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:48:40,251::task::595::TaskManager.Task::(_updateState) Task=`3b3870f7-9c69-4b9c-b5df-f5d4ca2b3e91`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:48:40,253::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:48:40,254::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:48:40,256::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:48:40,258::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:48:40,259::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:48:40,260::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:48:40,262::task::990::TaskManager.Task::(_decref) Task=`3b3870f7-9c69-4b9c-b5df-f5d4ca2b3e91`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-05-13 12:48:40,697::vmChannels::102::vds::(_do_add_channels) fileno 185 was added to unconnected channels. >VM Channels Listener::DEBUG::2014-05-13 12:48:40,697::vmChannels::128::vds::(_handle_unconnected) Trying to connect fileno 185. >VM Channels Listener::DEBUG::2014-05-13 12:48:40,697::guestIF::194::vm.Vm::(_connect) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Attempting connection to /var/lib/libvirt/qemu/channels/4da6d3ed-e2fd-4318-a58f-715cc1756430.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-05-13 12:48:40,700::guestIF::197::vm.Vm::(_connect) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Connected to /var/lib/libvirt/qemu/channels/4da6d3ed-e2fd-4318-a58f-715cc1756430.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-05-13 12:48:40,700::vmChannels::144::vds::(_handle_unconnected) Connecting to fileno 185 succeeded. >Thread-357::DEBUG::2014-05-13 12:48:40,855::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:48:40,865::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000824914 s, 474 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:48:41,831::BindingXMLRPC::1067::vds::(wrapper) client [10.35.163.66]::call vmGetStats with ('4da6d3ed-e2fd-4318-a58f-715cc1756430',) {} >Thread-66::DEBUG::2014-05-13 12:48:41,832::vm::587::vm.Vm::(_getCpuStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::CPU stats not available: 'NoneType' object is unsubscriptable >Thread-66::DEBUG::2014-05-13 12:48:41,832::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc stats not available >Thread-66::DEBUG::2014-05-13 12:48:41,832::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda stats not available >Thread-66::DEBUG::2014-05-13 12:48:41,832::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc latency not available >Thread-66::DEBUG::2014-05-13 12:48:41,833::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda latency not available >Thread-66::DEBUG::2014-05-13 12:48:41,834::BindingXMLRPC::1074::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'guestFQDN': '', 'pid': '32084', 'displayIp': '10.35.116.3', 'displayPort': u'5900', 'session': 'Unknown', 'displaySecurePort': u'5901', 'timeOffset': '0', 'hash': '-120934615822187974', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '1048576', 'balloon_cur': '1048576', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {}, 'vmId': '4da6d3ed-e2fd-4318-a58f-715cc1756430', 'displayType': 'qxl', 'cpuUser': '0.00', 'disks': {u'hdc': {'truesize': '0', 'apparentsize': '0'}, u'sda': {'truesize': '24576', 'apparentsize': '5368709120', 'imageID': 'ac82bfa3-8cd5-4643-b36a-5ad28040fe0a'}}, 'monitorResponse': '0', 'statsAge': '1.79', 'elapsedTime': '2', 'vmType': 'kvm', 'cpuSys': '0.00', 'guestCPUCount': -1, 'appsList': [], 'guestIPs': ''}]} >BindingXMLRPC::ERROR::2014-05-13 12:48:42,233::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:48:42,385::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-344::DEBUG::2014-05-13 12:48:42,512::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:48:42,532::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000411457 s, 846 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:48:42,562::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0394162 s, 26.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:48:44,575::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 12:48:44,892::task::595::TaskManager.Task::(_updateState) Task=`852000df-6743-487d-a988-96d738cff033`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:48:44,892::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:48:44,892::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000824914', 'lastCheck': '4.0', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000842825', 'lastCheck': '7.9', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000411457', 'lastCheck': '2.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:48:44,893::task::1185::TaskManager.Task::(prepare) Task=`852000df-6743-487d-a988-96d738cff033`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000824914', 'lastCheck': '4.0', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000842825', 'lastCheck': '7.9', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000411457', 'lastCheck': '2.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:48:44,893::task::595::TaskManager.Task::(_updateState) Task=`852000df-6743-487d-a988-96d738cff033`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:48:44,893::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:48:44,893::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:48:44,893::task::990::TaskManager.Task::(_decref) Task=`852000df-6743-487d-a988-96d738cff033`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:48:44,911::vm::587::vm.Vm::(_getCpuStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::CPU stats not available: 'NoneType' object is unsubscriptable >Thread-66::DEBUG::2014-05-13 12:48:44,912::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc stats not available >Thread-66::DEBUG::2014-05-13 12:48:44,912::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda stats not available >Thread-66::DEBUG::2014-05-13 12:48:44,912::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc latency not available >Thread-66::DEBUG::2014-05-13 12:48:44,912::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda latency not available >Dummy-337::DEBUG::2014-05-13 12:48:44,987::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.369491 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:48:45,252::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:45,328::vm::587::vm.Vm::(_getCpuStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::CPU stats not available: 'NoneType' object is unsubscriptable >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:45,328::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc stats not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:45,329::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda stats not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:45,329::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc latency not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:45,329::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda latency not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:45,337::vm::587::vm.Vm::(_getCpuStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::CPU stats not available: 'NoneType' object is unsubscriptable >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:45,337::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc stats not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:45,337::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda stats not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:45,337::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc latency not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:45,337::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda latency not available >Thread-322::DEBUG::2014-05-13 12:48:46,962::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:48:46,971::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000562065 s, 7.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:48:47,000::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:48:47,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353002 s, 29.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:48:48,015::BindingXMLRPC::1067::vds::(wrapper) client [10.35.163.66]::call vmGetStats with ('4da6d3ed-e2fd-4318-a58f-715cc1756430',) {} >Thread-66::DEBUG::2014-05-13 12:48:48,015::vm::587::vm.Vm::(_getCpuStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::CPU stats not available: 'NoneType' object is unsubscriptable >Thread-66::DEBUG::2014-05-13 12:48:48,015::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc stats not available >Thread-66::DEBUG::2014-05-13 12:48:48,016::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda stats not available >Thread-66::DEBUG::2014-05-13 12:48:48,016::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc latency not available >Thread-66::DEBUG::2014-05-13 12:48:48,016::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda latency not available >Thread-66::DEBUG::2014-05-13 12:48:48,017::BindingXMLRPC::1074::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'guestFQDN': '', 'pid': '32084', 'displayIp': '10.35.116.3', 'displayPort': u'5900', 'session': 'Unknown', 'displaySecurePort': u'5901', 'timeOffset': '0', 'hash': '-120934615822187974', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '1048576', 'balloon_cur': '1048576', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {}, 'vmId': '4da6d3ed-e2fd-4318-a58f-715cc1756430', 'displayType': 'qxl', 'cpuUser': '0.00', 'disks': {u'hdc': {'truesize': '0', 'apparentsize': '0'}, u'sda': {'truesize': '24576', 'apparentsize': '5368709120', 'imageID': 'ac82bfa3-8cd5-4643-b36a-5ad28040fe0a'}}, 'monitorResponse': '0', 'statsAge': '1.85', 'elapsedTime': '8', 'vmType': 'kvm', 'cpuSys': '0.00', 'guestCPUCount': -1, 'appsList': [], 'guestIPs': ''}]} >BindingXMLRPC::ERROR::2014-05-13 12:48:48,270::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:48:49,090::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:48:49,165::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311378 s, 32.9 MB/s\n'; <rc> = 0 >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:50,344::vm::587::vm.Vm::(_getCpuStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::CPU stats not available: 'NoneType' object is unsubscriptable >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:50,344::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc stats not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:50,344::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda stats not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:50,344::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc latency not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:50,344::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda latency not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:50,346::vm::587::vm.Vm::(_getCpuStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::CPU stats not available: 'NoneType' object is unsubscriptable >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:50,346::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc stats not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:50,346::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda stats not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:50,346::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc latency not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:50,346::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda latency not available >Thread-66::DEBUG::2014-05-13 12:48:50,369::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:48:50,369::task::595::TaskManager.Task::(_updateState) Task=`59d022e6-bc09-4acc-8c73-53184bf275f6`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:48:50,369::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:48:50,370::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:48:50,370::task::1185::TaskManager.Task::(prepare) Task=`59d022e6-bc09-4acc-8c73-53184bf275f6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:48:50,370::task::595::TaskManager.Task::(_updateState) Task=`59d022e6-bc09-4acc-8c73-53184bf275f6`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:48:50,370::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:48:50,370::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:48:50,371::task::990::TaskManager.Task::(_decref) Task=`59d022e6-bc09-4acc-8c73-53184bf275f6`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:48:50,415::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:48:50,416::task::595::TaskManager.Task::(_updateState) Task=`98807574-2b2a-4633-953b-a0b31e13b938`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:48:50,416::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:48:50,416::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b66f278d-37d4-4032-bce2-379c263b2f97`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:48:50,417::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:48:50,417::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:48:50,417::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b66f278d-37d4-4032-bce2-379c263b2f97`::Granted request >Thread-338::DEBUG::2014-05-13 12:48:50,417::task::827::TaskManager.Task::(resourceAcquired) Task=`98807574-2b2a-4633-953b-a0b31e13b938`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:48:50,417::task::990::TaskManager.Task::(_decref) Task=`98807574-2b2a-4633-953b-a0b31e13b938`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:48:50,418::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923394306048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:48:50,419::task::1185::TaskManager.Task::(prepare) Task=`98807574-2b2a-4633-953b-a0b31e13b938`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923394306048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:48:50,419::task::595::TaskManager.Task::(_updateState) Task=`98807574-2b2a-4633-953b-a0b31e13b938`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:48:50,419::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:48:50,419::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:48:50,419::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:48:50,420::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:48:50,420::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:48:50,420::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:48:50,420::task::990::TaskManager.Task::(_decref) Task=`98807574-2b2a-4633-953b-a0b31e13b938`::ref 0 aborting False >Thread-357::DEBUG::2014-05-13 12:48:50,877::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:48:50,886::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000373377 s, 1.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:48:51,037::BindingXMLRPC::1067::vds::(wrapper) client [10.35.163.66]::call vmGetStats with ('4da6d3ed-e2fd-4318-a58f-715cc1756430',) {} >Thread-66::DEBUG::2014-05-13 12:48:51,037::vm::587::vm.Vm::(_getCpuStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::CPU stats not available: 'NoneType' object is unsubscriptable >Thread-66::DEBUG::2014-05-13 12:48:51,037::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc stats not available >Thread-66::DEBUG::2014-05-13 12:48:51,037::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda stats not available >Thread-66::DEBUG::2014-05-13 12:48:51,038::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc latency not available >Thread-66::DEBUG::2014-05-13 12:48:51,038::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda latency not available >Thread-66::DEBUG::2014-05-13 12:48:51,039::BindingXMLRPC::1074::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'guestFQDN': '', 'pid': '32084', 'displayIp': '10.35.116.3', 'displayPort': u'5900', 'session': 'Unknown', 'displaySecurePort': u'5901', 'timeOffset': '0', 'hash': '-120934615822187974', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '1048576', 'balloon_cur': '1048576', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {}, 'vmId': '4da6d3ed-e2fd-4318-a58f-715cc1756430', 'displayType': 'qxl', 'cpuUser': '0.00', 'disks': {u'hdc': {'truesize': '0', 'apparentsize': '0'}, u'sda': {'truesize': '24576', 'apparentsize': '5368709120', 'imageID': 'ac82bfa3-8cd5-4643-b36a-5ad28040fe0a'}}, 'monitorResponse': '0', 'statsAge': '0.87', 'elapsedTime': '11', 'vmType': 'kvm', 'cpuSys': '0.00', 'guestCPUCount': -1, 'appsList': [], 'guestIPs': ''}]} >Dummy-337::DEBUG::2014-05-13 12:48:51,179::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:48:51,255::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0350668 s, 29.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:48:51,289::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-344::DEBUG::2014-05-13 12:48:52,548::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:48:52,558::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000403237 s, 863 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:48:53,267::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:48:53,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0354797 s, 28.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:48:54,098::BindingXMLRPC::1067::vds::(wrapper) client [10.35.163.66]::call vmGetStats with ('4da6d3ed-e2fd-4318-a58f-715cc1756430',) {} >Thread-66::DEBUG::2014-05-13 12:48:54,098::vm::587::vm.Vm::(_getCpuStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::CPU stats not available: 'NoneType' object is unsubscriptable >Thread-66::DEBUG::2014-05-13 12:48:54,098::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc stats not available >Thread-66::DEBUG::2014-05-13 12:48:54,098::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda stats not available >Thread-66::DEBUG::2014-05-13 12:48:54,098::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc latency not available >Thread-66::DEBUG::2014-05-13 12:48:54,099::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda latency not available >Thread-66::DEBUG::2014-05-13 12:48:54,100::BindingXMLRPC::1074::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'guestFQDN': '', 'pid': '32084', 'displayIp': '10.35.116.3', 'displayPort': u'5900', 'session': 'Unknown', 'displaySecurePort': u'5901', 'timeOffset': '0', 'hash': '-120934615822187974', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '1048576', 'balloon_cur': '1048576', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {}, 'vmId': '4da6d3ed-e2fd-4318-a58f-715cc1756430', 'displayType': 'qxl', 'cpuUser': '0.00', 'disks': {u'hdc': {'truesize': '0', 'apparentsize': '0'}, u'sda': {'truesize': '24576', 'apparentsize': '5368709120', 'imageID': 'ac82bfa3-8cd5-4643-b36a-5ad28040fe0a'}}, 'monitorResponse': '0', 'statsAge': '1.93', 'elapsedTime': '14', 'vmType': 'kvm', 'cpuSys': '0.00', 'guestCPUCount': -1, 'appsList': [], 'guestIPs': ''}]} >BindingXMLRPC::ERROR::2014-05-13 12:48:54,306::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:55,353::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc stats not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:55,353::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda stats not available >Dummy-337::DEBUG::2014-05-13 12:48:55,357::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:55,358::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc latency not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:55,365::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda latency not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:55,367::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc stats not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:55,368::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda stats not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:55,368::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc latency not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:48:55,368::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda latency not available >Dummy-337::DEBUG::2014-05-13 12:48:55,432::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0308797 s, 33.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:48:56,980::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:48:56,988::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000747104 s, 5.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:48:57,158::BindingXMLRPC::1067::vds::(wrapper) client [10.35.163.66]::call vmGetStats with ('4da6d3ed-e2fd-4318-a58f-715cc1756430',) {} >Thread-66::DEBUG::2014-05-13 12:48:57,159::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc stats not available >Thread-66::DEBUG::2014-05-13 12:48:57,159::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda stats not available >Thread-66::DEBUG::2014-05-13 12:48:57,159::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc latency not available >Thread-66::DEBUG::2014-05-13 12:48:57,159::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda latency not available >Thread-66::DEBUG::2014-05-13 12:48:57,160::BindingXMLRPC::1074::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'guestFQDN': '', 'pid': '32084', 'displayIp': '10.35.116.3', 'displayPort': u'5900', 'session': 'Unknown', 'displaySecurePort': u'5901', 'timeOffset': '0', 'hash': '-120934615822187974', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '1048576', 'balloon_cur': '1048576', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {}, 'vmId': '4da6d3ed-e2fd-4318-a58f-715cc1756430', 'displayType': 'qxl', 'cpuUser': '3.14', 'disks': {u'hdc': {'truesize': '0', 'apparentsize': '0'}, u'sda': {'truesize': '24576', 'apparentsize': '5368709120', 'imageID': 'ac82bfa3-8cd5-4643-b36a-5ad28040fe0a'}}, 'monitorResponse': '0', 'statsAge': '0.99', 'elapsedTime': '17', 'vmType': 'kvm', 'cpuSys': '4.17', 'guestCPUCount': -1, 'appsList': [], 'guestIPs': ''}]} >BindingXMLRPC::ERROR::2014-05-13 12:48:57,327::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:48:57,445::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:48:57,527::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0371025 s, 27.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:48:59,540::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:48:59,630::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0472573 s, 21.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:49:00,178::task::595::TaskManager.Task::(_updateState) Task=`516186be-d2a0-41f9-ba92-56b99a4c53ed`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:49:00,178::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:49:00,178::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000373377', 'lastCheck': '9.3', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000747104', 'lastCheck': '3.2', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000403237', 'lastCheck': '7.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:49:00,179::task::1185::TaskManager.Task::(prepare) Task=`516186be-d2a0-41f9-ba92-56b99a4c53ed`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000373377', 'lastCheck': '9.3', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000747104', 'lastCheck': '3.2', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000403237', 'lastCheck': '7.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:49:00,179::task::595::TaskManager.Task::(_updateState) Task=`516186be-d2a0-41f9-ba92-56b99a4c53ed`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:49:00,179::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:49:00,179::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:49:00,179::task::990::TaskManager.Task::(_decref) Task=`516186be-d2a0-41f9-ba92-56b99a4c53ed`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:49:00,197::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc stats not available >Thread-66::DEBUG::2014-05-13 12:49:00,197::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda stats not available >Thread-66::DEBUG::2014-05-13 12:49:00,197::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc latency not available >Thread-66::DEBUG::2014-05-13 12:49:00,197::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda latency not available >BindingXMLRPC::ERROR::2014-05-13 12:49:00,345::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:00,375::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc stats not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:00,375::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda stats not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:00,381::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc latency not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:00,381::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda latency not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:00,382::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc stats not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:00,383::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda stats not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:00,383::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc latency not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:00,383::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda latency not available >Thread-66::DEBUG::2014-05-13 12:49:00,467::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:49:00,467::task::595::TaskManager.Task::(_updateState) Task=`ac46df3e-c7b2-4fd7-8c4f-22ad7e397328`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:49:00,468::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:49:00,468::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:49:00,468::task::1185::TaskManager.Task::(prepare) Task=`ac46df3e-c7b2-4fd7-8c4f-22ad7e397328`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:49:00,468::task::595::TaskManager.Task::(_updateState) Task=`ac46df3e-c7b2-4fd7-8c4f-22ad7e397328`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:49:00,468::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:49:00,469::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:49:00,469::task::990::TaskManager.Task::(_decref) Task=`ac46df3e-c7b2-4fd7-8c4f-22ad7e397328`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:49:00,513::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:49:00,514::task::595::TaskManager.Task::(_updateState) Task=`68a07240-3e87-4cbb-913d-4b1e2b7c1fc5`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:49:00,514::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:49:00,514::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`766e401c-f837-4737-ae63-e4dd807f3990`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:49:00,514::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:49:00,515::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:49:00,515::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`766e401c-f837-4737-ae63-e4dd807f3990`::Granted request >Thread-338::DEBUG::2014-05-13 12:49:00,515::task::827::TaskManager.Task::(resourceAcquired) Task=`68a07240-3e87-4cbb-913d-4b1e2b7c1fc5`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:49:00,515::task::990::TaskManager.Task::(_decref) Task=`68a07240-3e87-4cbb-913d-4b1e2b7c1fc5`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:49:00,516::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923394306048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:49:00,516::task::1185::TaskManager.Task::(prepare) Task=`68a07240-3e87-4cbb-913d-4b1e2b7c1fc5`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923394306048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:49:00,517::task::595::TaskManager.Task::(_updateState) Task=`68a07240-3e87-4cbb-913d-4b1e2b7c1fc5`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:49:00,517::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:49:00,517::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:49:00,517::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:49:00,518::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:49:00,518::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:49:00,518::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:49:00,518::task::990::TaskManager.Task::(_decref) Task=`68a07240-3e87-4cbb-913d-4b1e2b7c1fc5`::ref 0 aborting False >Thread-357::DEBUG::2014-05-13 12:49:00,897::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:49:00,909::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.00104937 s, 373 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:49:01,643::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:49:01,722::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035459 s, 28.9 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:49:02,569::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:49:02,578::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000496024 s, 702 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:49:03,266::BindingXMLRPC::1067::vds::(wrapper) client [10.35.163.66]::call vmGetStats with ('4da6d3ed-e2fd-4318-a58f-715cc1756430',) {} flowID [3e537ec9] >Thread-66::DEBUG::2014-05-13 12:49:03,266::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc stats not available >Thread-66::DEBUG::2014-05-13 12:49:03,266::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda stats not available >Thread-66::DEBUG::2014-05-13 12:49:03,266::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc latency not available >Thread-66::DEBUG::2014-05-13 12:49:03,267::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda latency not available >Thread-66::DEBUG::2014-05-13 12:49:03,268::BindingXMLRPC::1074::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'guestFQDN': '', 'pid': '32084', 'displayIp': '10.35.116.3', 'displayPort': u'5900', 'session': 'Unknown', 'displaySecurePort': u'5901', 'timeOffset': '0', 'hash': '-120934615822187974', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '1048576', 'balloon_cur': '1048576', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {}, 'vmId': '4da6d3ed-e2fd-4318-a58f-715cc1756430', 'displayType': 'qxl', 'cpuUser': '3.14', 'disks': {u'hdc': {'truesize': '0', 'apparentsize': '0'}, u'sda': {'truesize': '24576', 'apparentsize': '5368709120', 'imageID': 'ac82bfa3-8cd5-4643-b36a-5ad28040fe0a'}}, 'monitorResponse': '0', 'statsAge': '1.09', 'elapsedTime': '23', 'vmType': 'kvm', 'cpuSys': '4.17', 'guestCPUCount': -1, 'appsList': [], 'guestIPs': ''}]} >BindingXMLRPC::ERROR::2014-05-13 12:49:03,363::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:49:03,735::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:49:03,832::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0543457 s, 18.8 MB/s\n'; <rc> = 0 >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:05,389::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc stats not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:05,390::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda stats not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:05,394::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc latency not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:05,394::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda latency not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:05,395::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc stats not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:05,395::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda stats not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:05,396::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc latency not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:05,396::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda latency not available >Dummy-337::DEBUG::2014-05-13 12:49:05,845::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:49:05,925::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0367609 s, 27.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:49:06,327::BindingXMLRPC::1067::vds::(wrapper) client [10.35.163.66]::call vmGetStats with ('4da6d3ed-e2fd-4318-a58f-715cc1756430',) {} >Thread-66::DEBUG::2014-05-13 12:49:06,328::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc stats not available >Thread-66::DEBUG::2014-05-13 12:49:06,328::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda stats not available >Thread-66::DEBUG::2014-05-13 12:49:06,328::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc latency not available >Thread-66::DEBUG::2014-05-13 12:49:06,328::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda latency not available >Thread-66::DEBUG::2014-05-13 12:49:06,329::BindingXMLRPC::1074::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'guestFQDN': '', 'pid': '32084', 'displayIp': '10.35.116.3', 'displayPort': u'5900', 'session': 'Unknown', 'displaySecurePort': u'5901', 'timeOffset': '0', 'hash': '-120934615822187974', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '1048576', 'balloon_cur': '1048576', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {}, 'vmId': '4da6d3ed-e2fd-4318-a58f-715cc1756430', 'displayType': 'qxl', 'cpuUser': '3.14', 'disks': {u'hdc': {'truesize': '0', 'apparentsize': '0'}, u'sda': {'truesize': '24576', 'apparentsize': '5368709120', 'imageID': 'ac82bfa3-8cd5-4643-b36a-5ad28040fe0a'}}, 'monitorResponse': '0', 'statsAge': '0.16', 'elapsedTime': '26', 'vmType': 'kvm', 'cpuSys': '4.17', 'guestCPUCount': -1, 'appsList': [], 'guestIPs': ''}]} >BindingXMLRPC::ERROR::2014-05-13 12:49:06,382::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:49:06,996::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:49:07,004::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000664167 s, 6.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:49:07,938::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:49:08,017::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357209 s, 28.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:49:09,390::BindingXMLRPC::1067::vds::(wrapper) client [10.35.163.66]::call vmGetStats with ('4da6d3ed-e2fd-4318-a58f-715cc1756430',) {} flowID [72471f8f] >Thread-66::DEBUG::2014-05-13 12:49:09,390::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc stats not available >Thread-66::DEBUG::2014-05-13 12:49:09,390::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda stats not available >Thread-66::DEBUG::2014-05-13 12:49:09,390::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc latency not available >Thread-66::DEBUG::2014-05-13 12:49:09,391::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda latency not available >Thread-66::DEBUG::2014-05-13 12:49:09,392::BindingXMLRPC::1074::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'guestFQDN': '', 'pid': '32084', 'displayIp': '10.35.116.3', 'displayPort': u'5900', 'session': 'Unknown', 'displaySecurePort': u'5901', 'timeOffset': '0', 'hash': '-120934615822187974', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '1048576', 'balloon_cur': '1048576', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {}, 'vmId': '4da6d3ed-e2fd-4318-a58f-715cc1756430', 'displayType': 'qxl', 'cpuUser': '3.14', 'disks': {u'hdc': {'truesize': '0', 'apparentsize': '0'}, u'sda': {'truesize': '24576', 'apparentsize': '5368709120', 'imageID': 'ac82bfa3-8cd5-4643-b36a-5ad28040fe0a'}}, 'monitorResponse': '0', 'statsAge': '1.22', 'elapsedTime': '30', 'vmType': 'kvm', 'cpuSys': '4.17', 'guestCPUCount': -1, 'appsList': [], 'guestIPs': ''}]} >BindingXMLRPC::ERROR::2014-05-13 12:49:09,401::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:49:10,030::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:49:10,110::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373323 s, 27.4 MB/s\n'; <rc> = 0 >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:10,403::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc stats not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:10,403::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda stats not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:10,440::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc latency not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:10,440::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda latency not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:10,441::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc stats not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:10,441::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda stats not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:10,442::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc latency not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:10,442::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda latency not available >Thread-66::DEBUG::2014-05-13 12:49:10,641::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:49:10,641::task::595::TaskManager.Task::(_updateState) Task=`0319d40b-e36d-4e35-af48-2dbd498ef661`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:49:10,641::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:49:10,642::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:49:10,642::task::1185::TaskManager.Task::(prepare) Task=`0319d40b-e36d-4e35-af48-2dbd498ef661`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:49:10,642::task::595::TaskManager.Task::(_updateState) Task=`0319d40b-e36d-4e35-af48-2dbd498ef661`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:49:10,642::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:49:10,642::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:49:10,642::task::990::TaskManager.Task::(_decref) Task=`0319d40b-e36d-4e35-af48-2dbd498ef661`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:49:10,689::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:49:10,689::task::595::TaskManager.Task::(_updateState) Task=`1974a617-446e-40f8-83ec-dc0420a6533d`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:49:10,689::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:49:10,690::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a3f920c2-80a3-4e4e-acef-a03056d58d59`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:49:10,690::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:49:10,690::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:49:10,690::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a3f920c2-80a3-4e4e-acef-a03056d58d59`::Granted request >Thread-338::DEBUG::2014-05-13 12:49:10,691::task::827::TaskManager.Task::(resourceAcquired) Task=`1974a617-446e-40f8-83ec-dc0420a6533d`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:49:10,691::task::990::TaskManager.Task::(_decref) Task=`1974a617-446e-40f8-83ec-dc0420a6533d`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:49:10,692::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923394306048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:49:10,692::task::1185::TaskManager.Task::(prepare) Task=`1974a617-446e-40f8-83ec-dc0420a6533d`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923394306048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:49:10,692::task::595::TaskManager.Task::(_updateState) Task=`1974a617-446e-40f8-83ec-dc0420a6533d`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:49:10,692::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:49:10,692::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:49:10,693::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:49:10,693::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:49:10,693::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:49:10,693::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:49:10,693::task::990::TaskManager.Task::(_decref) Task=`1974a617-446e-40f8-83ec-dc0420a6533d`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-05-13 12:49:10,733::vmChannels::91::vds::(_handle_timeouts) Timeout on fileno 185. >Thread-357::DEBUG::2014-05-13 12:49:10,921::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:49:10,930::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000419981 s, 931 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:49:12,123::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:49:12,202::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357337 s, 28.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:49:12,410::BindingXMLRPC::1067::vds::(wrapper) client [10.35.163.66]::call vmGetStats with ('4da6d3ed-e2fd-4318-a58f-715cc1756430',) {} >Thread-66::DEBUG::2014-05-13 12:49:12,411::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc stats not available >Thread-66::DEBUG::2014-05-13 12:49:12,411::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda stats not available >Thread-66::DEBUG::2014-05-13 12:49:12,411::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc latency not available >Thread-66::DEBUG::2014-05-13 12:49:12,411::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda latency not available >Thread-66::DEBUG::2014-05-13 12:49:12,412::BindingXMLRPC::1074::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'guestFQDN': '', 'pid': '32084', 'displayIp': '10.35.116.3', 'displayPort': u'5900', 'session': 'Unknown', 'displaySecurePort': u'5901', 'timeOffset': '0', 'hash': '-120934615822187974', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '1048576', 'balloon_cur': '1048576', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {}, 'vmId': '4da6d3ed-e2fd-4318-a58f-715cc1756430', 'displayType': 'qxl', 'cpuUser': '0.36', 'disks': {u'hdc': {'truesize': '0', 'apparentsize': '0'}, u'sda': {'truesize': '24576', 'apparentsize': '5368709120', 'imageID': 'ac82bfa3-8cd5-4643-b36a-5ad28040fe0a'}}, 'monitorResponse': '0', 'statsAge': '0.23', 'elapsedTime': '33', 'vmType': 'kvm', 'cpuSys': '0.40', 'guestCPUCount': -1, 'appsList': [], 'guestIPs': ''}]} >BindingXMLRPC::ERROR::2014-05-13 12:49:12,428::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-344::DEBUG::2014-05-13 12:49:12,592::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:49:12,601::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000390479 s, 891 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:49:14,215::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:49:14,297::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384009 s, 26.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:49:15,442::task::595::TaskManager.Task::(_updateState) Task=`c3c8860c-03ec-4579-989a-6940db24a9b7`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:49:15,443::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:49:15,443::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000419981', 'lastCheck': '4.5', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000664167', 'lastCheck': '8.4', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000390479', 'lastCheck': '2.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:49:15,443::task::1185::TaskManager.Task::(prepare) Task=`c3c8860c-03ec-4579-989a-6940db24a9b7`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000419981', 'lastCheck': '4.5', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000664167', 'lastCheck': '8.4', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000390479', 'lastCheck': '2.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:49:15,443::task::595::TaskManager.Task::(_updateState) Task=`c3c8860c-03ec-4579-989a-6940db24a9b7`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:49:15,443::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:49:15,444::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:49:15,444::task::990::TaskManager.Task::(_decref) Task=`c3c8860c-03ec-4579-989a-6940db24a9b7`::ref 0 aborting False >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:15,449::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc stats not available >BindingXMLRPC::ERROR::2014-05-13 12:49:15,447::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:15,451::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda stats not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:15,452::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc latency not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:15,452::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda latency not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:15,453::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc stats not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:15,454::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda stats not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:15,454::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc latency not available >GuestMonitor-vm1::DEBUG::2014-05-13 12:49:15,454::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda latency not available >Thread-66::DEBUG::2014-05-13 12:49:15,482::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc stats not available >Thread-66::DEBUG::2014-05-13 12:49:15,482::vm::645::vm.Vm::(_getDiskStats) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda stats not available >Thread-66::DEBUG::2014-05-13 12:49:15,482::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk hdc latency not available >Thread-66::DEBUG::2014-05-13 12:49:15,482::vm::684::vm.Vm::(_getDiskLatency) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Disk sda latency not available >Thread-66::DEBUG::2014-05-13 12:49:16,096::BindingXMLRPC::1067::vds::(wrapper) client [10.35.163.66]::call vmDestroy with ('4da6d3ed-e2fd-4318-a58f-715cc1756430',) {} flowID [5ea96370] >Thread-66::INFO::2014-05-13 12:49:16,097::API::327::vds::(destroy) vmContainerLock acquired by vm 4da6d3ed-e2fd-4318-a58f-715cc1756430 >Thread-66::DEBUG::2014-05-13 12:49:16,097::vm::4650::vm.Vm::(destroy) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::destroy Called >Thread-66::INFO::2014-05-13 12:49:16,097::vm::4596::vm.Vm::(releaseVm) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Release VM resources >Thread-66::DEBUG::2014-05-13 12:49:16,105::sampling::292::vm.Vm::(stop) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Stop statistics collection >Thread-66::DEBUG::2014-05-13 12:49:16,105::vmChannels::205::vds::(unregister) Delete fileno 185 from listener. >Thread-367::DEBUG::2014-05-13 12:49:16,105::sampling::323::vm.Vm::(run) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Stats thread finished >Dummy-337::DEBUG::2014-05-13 12:49:16,310::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:49:16,388::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317505 s, 32.3 MB/s\n'; <rc> = 0 >libvirtEventLoop::DEBUG::2014-05-13 12:49:16,486::vm::5151::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::event Stopped detail 1 opaque None >libvirtEventLoop::INFO::2014-05-13 12:49:16,488::vm::2320::vm.Vm::(_onQemuDeath) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::underlying process disconnected >Thread-66::DEBUG::2014-05-13 12:49:16,488::task::595::TaskManager.Task::(_updateState) Task=`cc9ef0ab-bd8c-47ab-bd30-45034259c6ef`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:49:16,489::logUtils::44::dispatcher::(wrapper) Run and protect: teardownImage(sdUUID='572b0496-5eab-41cd-a976-4514f111898e', spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', imgUUID='ac82bfa3-8cd5-4643-b36a-5ad28040fe0a', volUUID=None) >Thread-66::DEBUG::2014-05-13 12:49:16,490::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.572b0496-5eab-41cd-a976-4514f111898e`ReqID=`3dcba758-e8f5-4040-80b2-dd432c4e2dba`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '3296' at 'teardownImage' >Thread-66::DEBUG::2014-05-13 12:49:16,490::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' for lock type 'shared' >Thread-66::DEBUG::2014-05-13 12:49:16,490::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' is free. Now locking as 'shared' (1 active user) >Thread-66::DEBUG::2014-05-13 12:49:16,490::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.572b0496-5eab-41cd-a976-4514f111898e`ReqID=`3dcba758-e8f5-4040-80b2-dd432c4e2dba`::Granted request >Thread-66::DEBUG::2014-05-13 12:49:16,491::task::827::TaskManager.Task::(resourceAcquired) Task=`cc9ef0ab-bd8c-47ab-bd30-45034259c6ef`::_resourcesAcquired: Storage.572b0496-5eab-41cd-a976-4514f111898e (shared) >Thread-66::DEBUG::2014-05-13 12:49:16,491::task::990::TaskManager.Task::(_decref) Task=`cc9ef0ab-bd8c-47ab-bd30-45034259c6ef`::ref 1 aborting False >Thread-66::INFO::2014-05-13 12:49:16,491::logUtils::47::dispatcher::(wrapper) Run and protect: teardownImage, Return response: None >Thread-66::DEBUG::2014-05-13 12:49:16,491::task::1185::TaskManager.Task::(prepare) Task=`cc9ef0ab-bd8c-47ab-bd30-45034259c6ef`::finished: None >Thread-66::DEBUG::2014-05-13 12:49:16,491::task::595::TaskManager.Task::(_updateState) Task=`cc9ef0ab-bd8c-47ab-bd30-45034259c6ef`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:49:16,491::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.572b0496-5eab-41cd-a976-4514f111898e': < ResourceRef 'Storage.572b0496-5eab-41cd-a976-4514f111898e', isValid: 'True' obj: 'None'>} >Thread-66::DEBUG::2014-05-13 12:49:16,492::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:49:16,492::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' >Thread-66::DEBUG::2014-05-13 12:49:16,492::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' (0 active users) >Thread-66::DEBUG::2014-05-13 12:49:16,492::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' is free, finding out if anyone is waiting for it. >Thread-66::DEBUG::2014-05-13 12:49:16,493::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e', Clearing records. >Thread-66::DEBUG::2014-05-13 12:49:16,493::task::990::TaskManager.Task::(_decref) Task=`cc9ef0ab-bd8c-47ab-bd30-45034259c6ef`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:49:16,494::task::595::TaskManager.Task::(_updateState) Task=`3e5e6452-cff3-484c-a9c9-9e0bb82cccc6`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:49:16,494::logUtils::44::dispatcher::(wrapper) Run and protect: inappropriateDevices(thiefId='4da6d3ed-e2fd-4318-a58f-715cc1756430') >Thread-66::INFO::2014-05-13 12:49:16,500::logUtils::47::dispatcher::(wrapper) Run and protect: inappropriateDevices, Return response: None >Thread-66::DEBUG::2014-05-13 12:49:16,500::task::1185::TaskManager.Task::(prepare) Task=`3e5e6452-cff3-484c-a9c9-9e0bb82cccc6`::finished: None >Thread-66::DEBUG::2014-05-13 12:49:16,500::task::595::TaskManager.Task::(_updateState) Task=`3e5e6452-cff3-484c-a9c9-9e0bb82cccc6`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:49:16,500::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:49:16,501::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:49:16,501::task::990::TaskManager.Task::(_decref) Task=`3e5e6452-cff3-484c-a9c9-9e0bb82cccc6`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:49:16,501::vm::4644::vm.Vm::(deleteVm) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Total desktops after destroy of 4da6d3ed-e2fd-4318-a58f-715cc1756430 is 0 >Thread-66::DEBUG::2014-05-13 12:49:16,502::BindingXMLRPC::1074::vds::(wrapper) return vmDestroy with {'status': {'message': 'Machine destroyed', 'code': 0}} >libvirtEventLoop::DEBUG::2014-05-13 12:49:16,502::libvirtconnection::124::root::(wrapper) Unknown libvirterror: ecode: 42 edom: 10 level: 2 message: Domain not found: no domain with matching uuid '4da6d3ed-e2fd-4318-a58f-715cc1756430' >libvirtEventLoop::DEBUG::2014-05-13 12:49:16,503::vm::2727::vm.Vm::(setDownStatus) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Changed state to Down: Admin shut down >libvirtEventLoop::DEBUG::2014-05-13 12:49:16,504::sampling::292::vm.Vm::(stop) vmId=`4da6d3ed-e2fd-4318-a58f-715cc1756430`::Stop statistics collection >libvirtEventLoop::DEBUG::2014-05-13 12:49:16,506::libvirtconnection::124::root::(wrapper) Unknown libvirterror: ecode: 42 edom: 10 level: 2 message: Domain not found: no domain with matching uuid '4da6d3ed-e2fd-4318-a58f-715cc1756430' >VM Channels Listener::DEBUG::2014-05-13 12:49:16,740::vmChannels::112::vds::(_do_del_channels) fileno 185 was removed from listener. >Thread-322::DEBUG::2014-05-13 12:49:17,013::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:49:17,021::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000739164 s, 5.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:49:18,401::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:49:18,473::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:49:18,487::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384423 s, 26.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:49:20,499::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:49:20,587::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0456602 s, 22.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:49:20,795::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [19876d15] >Thread-66::DEBUG::2014-05-13 12:49:20,796::task::595::TaskManager.Task::(_updateState) Task=`e21845e3-2a23-4c2f-a39d-c937837bda36`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:49:20,796::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:49:20,796::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:49:20,796::task::1185::TaskManager.Task::(prepare) Task=`e21845e3-2a23-4c2f-a39d-c937837bda36`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:49:20,797::task::595::TaskManager.Task::(_updateState) Task=`e21845e3-2a23-4c2f-a39d-c937837bda36`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:49:20,797::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:49:20,797::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:49:20,797::task::990::TaskManager.Task::(_decref) Task=`e21845e3-2a23-4c2f-a39d-c937837bda36`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:49:20,802::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [19876d15] >Thread-338::DEBUG::2014-05-13 12:49:20,802::task::595::TaskManager.Task::(_updateState) Task=`fc291b56-b92e-4cbf-8161-7439ba68ebc1`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:49:20,802::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:49:20,803::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`bea3afcd-1dad-415d-a2d2-b3cecff8f0f4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:49:20,803::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:49:20,803::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:49:20,804::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`bea3afcd-1dad-415d-a2d2-b3cecff8f0f4`::Granted request >Thread-338::DEBUG::2014-05-13 12:49:20,804::task::827::TaskManager.Task::(resourceAcquired) Task=`fc291b56-b92e-4cbf-8161-7439ba68ebc1`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:49:20,804::task::990::TaskManager.Task::(_decref) Task=`fc291b56-b92e-4cbf-8161-7439ba68ebc1`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:49:20,805::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923394306048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:49:20,805::task::1185::TaskManager.Task::(prepare) Task=`fc291b56-b92e-4cbf-8161-7439ba68ebc1`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923394306048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:49:20,805::task::595::TaskManager.Task::(_updateState) Task=`fc291b56-b92e-4cbf-8161-7439ba68ebc1`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:49:20,806::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:49:20,806::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:49:20,806::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:49:20,806::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:49:20,806::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:49:20,807::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:49:20,807::task::990::TaskManager.Task::(_decref) Task=`fc291b56-b92e-4cbf-8161-7439ba68ebc1`::ref 0 aborting False >Thread-357::DEBUG::2014-05-13 12:49:20,944::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:49:20,954::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000881634 s, 443 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:49:21,490::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:49:22,600::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-344::DEBUG::2014-05-13 12:49:22,618::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:49:22,636::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000525109 s, 663 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:49:22,718::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0608864 s, 16.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:49:24,508::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:49:24,736::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:49:24,816::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0414212 s, 24.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:49:26,824::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:49:26,899::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0328893 s, 31.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:49:27,030::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:49:27,038::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000643332 s, 6.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:49:27,697::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-338::DEBUG::2014-05-13 12:49:28,875::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6e0ac8a1] >Thread-338::DEBUG::2014-05-13 12:49:28,876::task::595::TaskManager.Task::(_updateState) Task=`6975e26b-13ef-4891-89c8-1d4d34e16f4a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:49:28,876::logUtils::44::dispatcher::(wrapper) Run and protect: copyImage(sdUUID='572b0496-5eab-41cd-a976-4514f111898e', spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', vmUUID='', srcImgUUID='ac82bfa3-8cd5-4643-b36a-5ad28040fe0a', srcVolUUID='b4c44053-1991-4c45-aea6-2bcb369e4b65', dstImgUUID='f35efc5c-cd67-4de9-b1c6-9e08bafcae46', dstVolUUID='c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a', description='Active VM', dstSdUUID='572b0496-5eab-41cd-a976-4514f111898e', volType=6, volFormat=5, preallocate=2, postZero='false', force='false') >Thread-338::DEBUG::2014-05-13 12:49:28,878::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for b4c44053-1991-4c45-aea6-2bcb369e4b65 >Thread-338::INFO::2014-05-13 12:49:28,886::image::285::Storage.Image::(isLegal) image ac82bfa3-8cd5-4643-b36a-5ad28040fe0a in domain 572b0496-5eab-41cd-a976-4514f111898e has vollist ['b4c44053-1991-4c45-aea6-2bcb369e4b65'] >Thread-338::DEBUG::2014-05-13 12:49:28,886::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for b4c44053-1991-4c45-aea6-2bcb369e4b65 >Thread-338::DEBUG::2014-05-13 12:49:28,900::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for b4c44053-1991-4c45-aea6-2bcb369e4b65 >Thread-338::DEBUG::2014-05-13 12:49:28,907::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for b4c44053-1991-4c45-aea6-2bcb369e4b65 >Dummy-337::DEBUG::2014-05-13 12:49:28,912::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-338::DEBUG::2014-05-13 12:49:28,922::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for b4c44053-1991-4c45-aea6-2bcb369e4b65 >Thread-338::INFO::2014-05-13 12:49:28,941::image::215::Storage.Image::(getChain) sdUUID=572b0496-5eab-41cd-a976-4514f111898e imgUUID=ac82bfa3-8cd5-4643-b36a-5ad28040fe0a chain=[<storage.fileVolume.FileVolume object at 0x7fa3b001da10>] >Thread-338::DEBUG::2014-05-13 12:49:28,951::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.572b0496-5eab-41cd-a976-4514f111898e`ReqID=`2b791ebf-e8c3-4a84-a380-7736574feff2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1792' at 'copyImage' >Thread-338::DEBUG::2014-05-13 12:49:28,952::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:49:28,954::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:49:28,955::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.572b0496-5eab-41cd-a976-4514f111898e`ReqID=`2b791ebf-e8c3-4a84-a380-7736574feff2`::Granted request >Thread-338::DEBUG::2014-05-13 12:49:28,957::task::827::TaskManager.Task::(resourceAcquired) Task=`6975e26b-13ef-4891-89c8-1d4d34e16f4a`::_resourcesAcquired: Storage.572b0496-5eab-41cd-a976-4514f111898e (shared) >Thread-338::DEBUG::2014-05-13 12:49:28,958::task::990::TaskManager.Task::(_decref) Task=`6975e26b-13ef-4891-89c8-1d4d34e16f4a`::ref 1 aborting False >Thread-338::DEBUG::2014-05-13 12:49:28,962::task::752::TaskManager.Task::(_save) Task=`6975e26b-13ef-4891-89c8-1d4d34e16f4a`::_save: orig /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/6975e26b-13ef-4891-89c8-1d4d34e16f4a temp /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/6975e26b-13ef-4891-89c8-1d4d34e16f4a.temp >Thread-338::DEBUG::2014-05-13 12:49:28,982::taskManager::68::TaskManager::(scheduleJob) scheduled job copyImage_f35efc5c-cd67-4de9-b1c6-9e08bafcae46 for task 6975e26b-13ef-4891-89c8-1d4d34e16f4a >Thread-338::INFO::2014-05-13 12:49:28,983::logUtils::47::dispatcher::(wrapper) Run and protect: copyImage, Return response: None >Thread-338::DEBUG::2014-05-13 12:49:28,984::task::1180::TaskManager.Task::(prepare) Task=`6975e26b-13ef-4891-89c8-1d4d34e16f4a`::Prepare: 1 jobs exist, move to acquiring >Thread-338::DEBUG::2014-05-13 12:49:28,985::task::595::TaskManager.Task::(_updateState) Task=`6975e26b-13ef-4891-89c8-1d4d34e16f4a`::moving from state preparing -> state acquiring >Thread-338::DEBUG::2014-05-13 12:49:28,987::task::752::TaskManager.Task::(_save) Task=`6975e26b-13ef-4891-89c8-1d4d34e16f4a`::_save: orig /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/6975e26b-13ef-4891-89c8-1d4d34e16f4a temp /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/6975e26b-13ef-4891-89c8-1d4d34e16f4a.temp >Thread-338::DEBUG::2014-05-13 12:49:29,006::task::595::TaskManager.Task::(_updateState) Task=`6975e26b-13ef-4891-89c8-1d4d34e16f4a`::moving from state acquiring -> state queued >Thread-338::DEBUG::2014-05-13 12:49:29,008::task::752::TaskManager.Task::(_save) Task=`6975e26b-13ef-4891-89c8-1d4d34e16f4a`::_save: orig /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/6975e26b-13ef-4891-89c8-1d4d34e16f4a temp /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/6975e26b-13ef-4891-89c8-1d4d34e16f4a.temp >Thread-338::DEBUG::2014-05-13 12:49:29,024::taskManager::50::TaskManager::(_queueTask) queuing task: 6975e26b-13ef-4891-89c8-1d4d34e16f4a >Thread-338::DEBUG::2014-05-13 12:49:29,025::taskManager::56::TaskManager::(_queueTask) task queued: 6975e26b-13ef-4891-89c8-1d4d34e16f4a >0a261e11-bfd8-4d9b-aa38-2cf3d03bc34e::DEBUG::2014-05-13 12:49:29,025::threadPool::57::Misc.ThreadPool::(setRunningTask) Number of running tasks: 1 >Thread-338::DEBUG::2014-05-13 12:49:29,026::task::1182::TaskManager.Task::(prepare) Task=`6975e26b-13ef-4891-89c8-1d4d34e16f4a`::returning >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,027::threadPool::200::Misc.ThreadPool.WorkerThread::(_processNextTask) Task: 6975e26b-13ef-4891-89c8-1d4d34e16f4a running: <bound method Task.commit of <storage.task.Task instance at 0x7fa3740914d0>> with: None >Thread-338::DEBUG::2014-05-13 12:49:29,028::task::990::TaskManager.Task::(_decref) Task=`6975e26b-13ef-4891-89c8-1d4d34e16f4a`::ref 0 aborting False >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,028::task::1194::TaskManager.Task::(commit) Task=`6975e26b-13ef-4891-89c8-1d4d34e16f4a`::committing task: 6975e26b-13ef-4891-89c8-1d4d34e16f4a >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,030::task::595::TaskManager.Task::(_updateState) Task=`6975e26b-13ef-4891-89c8-1d4d34e16f4a`::moving from state queued -> state running >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,034::task::752::TaskManager.Task::(_save) Task=`6975e26b-13ef-4891-89c8-1d4d34e16f4a`::_save: orig /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/6975e26b-13ef-4891-89c8-1d4d34e16f4a temp /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/6975e26b-13ef-4891-89c8-1d4d34e16f4a.temp >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,051::task::905::TaskManager.Task::(_runJobs) Task=`6975e26b-13ef-4891-89c8-1d4d34e16f4a`::Task.run: running job 0: copyImage_f35efc5c-cd67-4de9-b1c6-9e08bafcae46: <bound method StoragePool.copyImage of <storage.sp.StoragePool object at 0x7fa3bc680ed0>> (args: ('572b0496-5eab-41cd-a976-4514f111898e', '', 'ac82bfa3-8cd5-4643-b36a-5ad28040fe0a', 'b4c44053-1991-4c45-aea6-2bcb369e4b65', 'f35efc5c-cd67-4de9-b1c6-9e08bafcae46', 'c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a', 'Active VM', '572b0496-5eab-41cd-a976-4514f111898e', 6, 5, 2, False, False) kwargs: {}) >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,051::task::331::TaskManager.Task::(run) Task=`6975e26b-13ef-4891-89c8-1d4d34e16f4a`::Job.run: running copyImage_f35efc5c-cd67-4de9-b1c6-9e08bafcae46: <bound method StoragePool.copyImage of <storage.sp.StoragePool object at 0x7fa3bc680ed0>> (args: ('572b0496-5eab-41cd-a976-4514f111898e', '', 'ac82bfa3-8cd5-4643-b36a-5ad28040fe0a', 'b4c44053-1991-4c45-aea6-2bcb369e4b65', 'f35efc5c-cd67-4de9-b1c6-9e08bafcae46', 'c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a', 'Active VM', '572b0496-5eab-41cd-a976-4514f111898e', 6, 5, 2, False, False) kwargs: {}) callback None >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,053::resourceManager::198::ResourceManager.Request::(__init__) ResName=`572b0496-5eab-41cd-a976-4514f111898e_imageNS.ac82bfa3-8cd5-4643-b36a-5ad28040fe0a`ReqID=`303eaafd-3b68-43e5-b1bf-3c9b189af561`::Request was made in '/usr/share/vdsm/storage/sp.py' line '1495' at 'copyImage' >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,054::resourceManager::542::ResourceManager::(registerResource) Trying to register resource '572b0496-5eab-41cd-a976-4514f111898e_imageNS.ac82bfa3-8cd5-4643-b36a-5ad28040fe0a' for lock type 'shared' >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,061::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for b4c44053-1991-4c45-aea6-2bcb369e4b65 >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,075::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for b4c44053-1991-4c45-aea6-2bcb369e4b65 >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,090::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for b4c44053-1991-4c45-aea6-2bcb369e4b65 >Dummy-337::DEBUG::2014-05-13 12:49:29,104::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.082569 s, 12.4 MB/s\n'; <rc> = 0 >6975e26b-13ef-4891-89c8-1d4d34e16f4a::INFO::2014-05-13 12:49:29,109::image::215::Storage.Image::(getChain) sdUUID=572b0496-5eab-41cd-a976-4514f111898e imgUUID=ac82bfa3-8cd5-4643-b36a-5ad28040fe0a chain=[<storage.fileVolume.FileVolume object at 0x7fa3bc653390>] >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,118::resourceManager::198::ResourceManager.Request::(__init__) ResName=`572b0496-5eab-41cd-a976-4514f111898e_volumeNS.b4c44053-1991-4c45-aea6-2bcb369e4b65`ReqID=`fa3240f0-eae8-4e08-915e-59862670b592`::Request was made in '/usr/share/vdsm/storage/resourceFactories.py' line '172' at '__getResourceCandidatesList' >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,119::resourceManager::542::ResourceManager::(registerResource) Trying to register resource '572b0496-5eab-41cd-a976-4514f111898e_volumeNS.b4c44053-1991-4c45-aea6-2bcb369e4b65' for lock type 'shared' >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,120::resourceManager::601::ResourceManager::(registerResource) Resource '572b0496-5eab-41cd-a976-4514f111898e_volumeNS.b4c44053-1991-4c45-aea6-2bcb369e4b65' is free. Now locking as 'shared' (1 active user) >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,121::resourceManager::238::ResourceManager.Request::(grant) ResName=`572b0496-5eab-41cd-a976-4514f111898e_volumeNS.b4c44053-1991-4c45-aea6-2bcb369e4b65`ReqID=`fa3240f0-eae8-4e08-915e-59862670b592`::Granted request >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,122::resourceManager::601::ResourceManager::(registerResource) Resource '572b0496-5eab-41cd-a976-4514f111898e_imageNS.ac82bfa3-8cd5-4643-b36a-5ad28040fe0a' is free. Now locking as 'shared' (1 active user) >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,123::resourceManager::238::ResourceManager.Request::(grant) ResName=`572b0496-5eab-41cd-a976-4514f111898e_imageNS.ac82bfa3-8cd5-4643-b36a-5ad28040fe0a`ReqID=`303eaafd-3b68-43e5-b1bf-3c9b189af561`::Granted request >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,125::resourceManager::198::ResourceManager.Request::(__init__) ResName=`572b0496-5eab-41cd-a976-4514f111898e_imageNS.f35efc5c-cd67-4de9-b1c6-9e08bafcae46`ReqID=`cd2e25e7-580a-48c0-bac7-c0d29ae519ad`::Request was made in '/usr/share/vdsm/storage/sp.py' line '1497' at 'copyImage' >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,126::resourceManager::542::ResourceManager::(registerResource) Trying to register resource '572b0496-5eab-41cd-a976-4514f111898e_imageNS.f35efc5c-cd67-4de9-b1c6-9e08bafcae46' for lock type 'exclusive' >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,129::resourceFactories::125::Storage.ResourcesFactories::(__getResourceCandidatesList) Image f35efc5c-cd67-4de9-b1c6-9e08bafcae46 does not exist in domain 572b0496-5eab-41cd-a976-4514f111898e >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,129::resourceManager::601::ResourceManager::(registerResource) Resource '572b0496-5eab-41cd-a976-4514f111898e_imageNS.f35efc5c-cd67-4de9-b1c6-9e08bafcae46' is free. Now locking as 'exclusive' (1 active user) >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,130::resourceManager::238::ResourceManager.Request::(grant) ResName=`572b0496-5eab-41cd-a976-4514f111898e_imageNS.f35efc5c-cd67-4de9-b1c6-9e08bafcae46`ReqID=`cd2e25e7-580a-48c0-bac7-c0d29ae519ad`::Granted request >6975e26b-13ef-4891-89c8-1d4d34e16f4a::INFO::2014-05-13 12:49:29,131::image::647::Storage.Image::(copyCollapsed) sdUUID=572b0496-5eab-41cd-a976-4514f111898e vmUUID= srcImgUUID=ac82bfa3-8cd5-4643-b36a-5ad28040fe0a srcVolUUID=b4c44053-1991-4c45-aea6-2bcb369e4b65 dstImgUUID=f35efc5c-cd67-4de9-b1c6-9e08bafcae46 dstVolUUID=c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a dstSdUUID=572b0496-5eab-41cd-a976-4514f111898e volType=6 volFormat=RAW preallocate=SPARSE force=False postZero=False >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,134::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for b4c44053-1991-4c45-aea6-2bcb369e4b65 >6975e26b-13ef-4891-89c8-1d4d34e16f4a::INFO::2014-05-13 12:49:29,186::image::697::Storage.Image::(copyCollapsed) copy source 572b0496-5eab-41cd-a976-4514f111898e:ac82bfa3-8cd5-4643-b36a-5ad28040fe0a:b4c44053-1991-4c45-aea6-2bcb369e4b65 vol size 10485760 destination 572b0496-5eab-41cd-a976-4514f111898e:f35efc5c-cd67-4de9-b1c6-9e08bafcae46:c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a apparentsize 10485760 >6975e26b-13ef-4891-89c8-1d4d34e16f4a::INFO::2014-05-13 12:49:29,186::image::285::Storage.Image::(isLegal) image f35efc5c-cd67-4de9-b1c6-9e08bafcae46 in domain 572b0496-5eab-41cd-a976-4514f111898e has vollist [] >6975e26b-13ef-4891-89c8-1d4d34e16f4a::INFO::2014-05-13 12:49:29,187::image::123::Storage.Image::(create) Create placeholder /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/572b0496-5eab-41cd-a976-4514f111898e/images/f35efc5c-cd67-4de9-b1c6-9e08bafcae46 for image's volumes >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,187::task::752::TaskManager.Task::(_save) Task=`6975e26b-13ef-4891-89c8-1d4d34e16f4a`::_save: orig /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/6975e26b-13ef-4891-89c8-1d4d34e16f4a temp /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/6975e26b-13ef-4891-89c8-1d4d34e16f4a.temp >6975e26b-13ef-4891-89c8-1d4d34e16f4a::INFO::2014-05-13 12:49:29,195::volume::443::Storage.Volume::(create) Creating volume c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,196::task::752::TaskManager.Task::(_save) Task=`6975e26b-13ef-4891-89c8-1d4d34e16f4a`::_save: orig /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/6975e26b-13ef-4891-89c8-1d4d34e16f4a temp /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/6975e26b-13ef-4891-89c8-1d4d34e16f4a.temp >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,204::task::752::TaskManager.Task::(_save) Task=`6975e26b-13ef-4891-89c8-1d4d34e16f4a`::_save: orig /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/6975e26b-13ef-4891-89c8-1d4d34e16f4a temp /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/6975e26b-13ef-4891-89c8-1d4d34e16f4a.temp >6975e26b-13ef-4891-89c8-1d4d34e16f4a::INFO::2014-05-13 12:49:29,219::fileVolume::144::Storage.Volume::(_create) Request to create RAW volume /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/572b0496-5eab-41cd-a976-4514f111898e/images/f35efc5c-cd67-4de9-b1c6-9e08bafcae46/c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a with size = 20480 sectors >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,221::task::752::TaskManager.Task::(_save) Task=`6975e26b-13ef-4891-89c8-1d4d34e16f4a`::_save: orig /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/6975e26b-13ef-4891-89c8-1d4d34e16f4a temp /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/6975e26b-13ef-4891-89c8-1d4d34e16f4a.temp >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,237::fileVolume::395::Storage.Volume::(newVolumeLease) Initializing volume lease volUUID=c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a sdUUID=572b0496-5eab-41cd-a976-4514f111898e, metaId=('/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/572b0496-5eab-41cd-a976-4514f111898e/images/f35efc5c-cd67-4de9-b1c6-9e08bafcae46/c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a',) >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,255::task::752::TaskManager.Task::(_save) Task=`6975e26b-13ef-4891-89c8-1d4d34e16f4a`::_save: orig /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/6975e26b-13ef-4891-89c8-1d4d34e16f4a temp /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/6975e26b-13ef-4891-89c8-1d4d34e16f4a.temp >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,262::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a >6975e26b-13ef-4891-89c8-1d4d34e16f4a::INFO::2014-05-13 12:49:29,274::volume::789::Storage.Volume::(prepare) Volume: preparing volume 572b0496-5eab-41cd-a976-4514f111898e/b4c44053-1991-4c45-aea6-2bcb369e4b65 >6975e26b-13ef-4891-89c8-1d4d34e16f4a::INFO::2014-05-13 12:49:29,293::volume::789::Storage.Volume::(prepare) Volume: preparing volume 572b0496-5eab-41cd-a976-4514f111898e/c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,336::volume::1094::Storage.Volume::(qemuConvert) (qemuConvert): COPY /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/572b0496-5eab-41cd-a976-4514f111898e/images/ac82bfa3-8cd5-4643-b36a-5ad28040fe0a/b4c44053-1991-4c45-aea6-2bcb369e4b65 (raw) to /rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/images/f35efc5c-cd67-4de9-b1c6-9e08bafcae46/c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a (raw) START >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,336::utils::683::Storage.Misc.excCmd::(watchCmd) '/bin/nice -n 19 /usr/bin/ionice -c 3 /usr/bin/qemu-img convert -t none -f raw /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/572b0496-5eab-41cd-a976-4514f111898e/images/ac82bfa3-8cd5-4643-b36a-5ad28040fe0a/b4c44053-1991-4c45-aea6-2bcb369e4b65 -O raw /rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/images/f35efc5c-cd67-4de9-b1c6-9e08bafcae46/c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a' (cwd None) >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:49:29,345::task::752::TaskManager.Task::(_save) Task=`6975e26b-13ef-4891-89c8-1d4d34e16f4a`::_save: orig /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/6975e26b-13ef-4891-89c8-1d4d34e16f4a temp /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/6975e26b-13ef-4891-89c8-1d4d34e16f4a.temp >Thread-66::DEBUG::2014-05-13 12:49:30,572::task::595::TaskManager.Task::(_updateState) Task=`ac670a96-46c8-4794-b379-88da5f13acd3`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:49:30,572::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:49:30,573::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000881634', 'lastCheck': '9.6', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000643332', 'lastCheck': '3.5', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000525109', 'lastCheck': '7.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:49:30,573::task::1185::TaskManager.Task::(prepare) Task=`ac670a96-46c8-4794-b379-88da5f13acd3`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000881634', 'lastCheck': '9.6', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000643332', 'lastCheck': '3.5', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000525109', 'lastCheck': '7.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:49:30,573::task::595::TaskManager.Task::(_updateState) Task=`ac670a96-46c8-4794-b379-88da5f13acd3`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:49:30,573::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:49:30,574::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:49:30,574::task::990::TaskManager.Task::(_decref) Task=`ac670a96-46c8-4794-b379-88da5f13acd3`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:49:30,716::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:49:30,860::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:49:30,860::task::595::TaskManager.Task::(_updateState) Task=`dd395d79-01c8-4830-8cb9-ba6191b75b89`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:49:30,860::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:49:30,861::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:49:30,861::task::1185::TaskManager.Task::(prepare) Task=`dd395d79-01c8-4830-8cb9-ba6191b75b89`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:49:30,861::task::595::TaskManager.Task::(_updateState) Task=`dd395d79-01c8-4830-8cb9-ba6191b75b89`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:49:30,861::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:49:30,861::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:49:30,862::task::990::TaskManager.Task::(_decref) Task=`dd395d79-01c8-4830-8cb9-ba6191b75b89`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:49:30,866::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:49:30,867::task::595::TaskManager.Task::(_updateState) Task=`331862b4-65ca-4c0e-8c3d-b0d0c5a8a2c3`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:49:30,867::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:49:30,868::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f2d200c3-6308-4a85-a9ae-7ef5c03f5ecc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:49:30,868::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:49:30,869::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:49:30,869::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f2d200c3-6308-4a85-a9ae-7ef5c03f5ecc`::Granted request >Thread-338::DEBUG::2014-05-13 12:49:30,870::task::827::TaskManager.Task::(resourceAcquired) Task=`331862b4-65ca-4c0e-8c3d-b0d0c5a8a2c3`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:49:30,870::task::990::TaskManager.Task::(_decref) Task=`331862b4-65ca-4c0e-8c3d-b0d0c5a8a2c3`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:49:30,871::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923394306048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:49:30,871::task::1185::TaskManager.Task::(prepare) Task=`331862b4-65ca-4c0e-8c3d-b0d0c5a8a2c3`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923394306048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:49:30,872::task::595::TaskManager.Task::(_updateState) Task=`331862b4-65ca-4c0e-8c3d-b0d0c5a8a2c3`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:49:30,872::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:49:30,872::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:49:30,873::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:49:30,873::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:49:30,873::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:49:30,873::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:49:30,874::task::990::TaskManager.Task::(_decref) Task=`331862b4-65ca-4c0e-8c3d-b0d0c5a8a2c3`::ref 0 aborting False >Thread-357::DEBUG::2014-05-13 12:49:30,966::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:49:30,977::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000518258 s, 754 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:49:31,136::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:49:31,505::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.324832 s, 3.2 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:49:32,079::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:49:32,104::task::595::TaskManager.Task::(_updateState) Task=`7e97df92-5d4a-424b-96aa-d756ea064578`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:49:32,104::logUtils::44::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-66::DEBUG::2014-05-13 12:49:32,105::taskManager::102::TaskManager::(getAllTasksStatuses) Entry. >Thread-66::DEBUG::2014-05-13 12:49:32,105::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 6975e26b-13ef-4891-89c8-1d4d34e16f4a >Thread-66::DEBUG::2014-05-13 12:49:32,105::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'} >Thread-66::DEBUG::2014-05-13 12:49:32,105::taskManager::113::TaskManager::(getAllTasksStatuses) Return: {'6975e26b-13ef-4891-89c8-1d4d34e16f4a': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'}} >Thread-66::INFO::2014-05-13 12:49:32,105::logUtils::47::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {'6975e26b-13ef-4891-89c8-1d4d34e16f4a': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'}}} >Thread-66::DEBUG::2014-05-13 12:49:32,105::task::1185::TaskManager.Task::(prepare) Task=`7e97df92-5d4a-424b-96aa-d756ea064578`::finished: {'allTasksStatus': {'6975e26b-13ef-4891-89c8-1d4d34e16f4a': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'}}} >Thread-66::DEBUG::2014-05-13 12:49:32,106::task::595::TaskManager.Task::(_updateState) Task=`7e97df92-5d4a-424b-96aa-d756ea064578`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:49:32,106::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:49:32,106::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:49:32,106::task::990::TaskManager.Task::(_decref) Task=`7e97df92-5d4a-424b-96aa-d756ea064578`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:49:32,774::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:49:32,795::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.00473042 s, 73.6 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:49:33,517::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:49:33,606::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0443801 s, 23.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:49:33,735::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:49:35,620::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:49:35,744::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0568318 s, 18.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:49:36,753::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:49:37,046::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:49:37,057::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000812008 s, 5.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:49:37,761::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:49:38,257::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.427903 s, 2.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:49:39,769::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:49:40,274::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:49:40,361::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0419196 s, 24.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:49:40,910::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5affc0d0] >Thread-66::DEBUG::2014-05-13 12:49:40,911::task::595::TaskManager.Task::(_updateState) Task=`67eb44a1-8e3a-4b81-911a-a71fb99dc2f1`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:49:40,911::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:49:40,912::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:49:40,912::task::1185::TaskManager.Task::(prepare) Task=`67eb44a1-8e3a-4b81-911a-a71fb99dc2f1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:49:40,912::task::595::TaskManager.Task::(_updateState) Task=`67eb44a1-8e3a-4b81-911a-a71fb99dc2f1`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:49:40,913::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:49:40,913::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:49:40,913::task::990::TaskManager.Task::(_decref) Task=`67eb44a1-8e3a-4b81-911a-a71fb99dc2f1`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:49:40,918::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5affc0d0] >Thread-338::DEBUG::2014-05-13 12:49:40,919::task::595::TaskManager.Task::(_updateState) Task=`ae7a4201-5867-432c-9628-a7c8052b8a54`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:49:40,919::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:49:40,919::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`23c29529-b70c-466a-8ac1-f00f289855fc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:49:40,919::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:49:40,920::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:49:40,920::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`23c29529-b70c-466a-8ac1-f00f289855fc`::Granted request >Thread-338::DEBUG::2014-05-13 12:49:40,920::task::827::TaskManager.Task::(resourceAcquired) Task=`ae7a4201-5867-432c-9628-a7c8052b8a54`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:49:40,920::task::990::TaskManager.Task::(_decref) Task=`ae7a4201-5867-432c-9628-a7c8052b8a54`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:49:40,921::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:49:40,921::task::1185::TaskManager.Task::(prepare) Task=`ae7a4201-5867-432c-9628-a7c8052b8a54`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:49:40,922::task::595::TaskManager.Task::(_updateState) Task=`ae7a4201-5867-432c-9628-a7c8052b8a54`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:49:40,922::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:49:40,922::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:49:40,922::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:49:40,923::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:49:40,923::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:49:40,923::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:49:40,923::task::990::TaskManager.Task::(_decref) Task=`ae7a4201-5867-432c-9628-a7c8052b8a54`::ref 0 aborting False >Thread-357::DEBUG::2014-05-13 12:49:40,990::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:49:41,000::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000455442 s, 859 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:49:42,113::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5c06d0d6] >Thread-66::DEBUG::2014-05-13 12:49:42,114::task::595::TaskManager.Task::(_updateState) Task=`7ad974cf-008c-412a-b4c6-b01bead3f50e`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:49:42,114::logUtils::44::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-66::DEBUG::2014-05-13 12:49:42,115::taskManager::102::TaskManager::(getAllTasksStatuses) Entry. >Thread-66::DEBUG::2014-05-13 12:49:42,115::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 6975e26b-13ef-4891-89c8-1d4d34e16f4a >Thread-66::DEBUG::2014-05-13 12:49:42,115::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'} >Thread-66::DEBUG::2014-05-13 12:49:42,115::taskManager::113::TaskManager::(getAllTasksStatuses) Return: {'6975e26b-13ef-4891-89c8-1d4d34e16f4a': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'}} >Thread-66::INFO::2014-05-13 12:49:42,116::logUtils::47::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {'6975e26b-13ef-4891-89c8-1d4d34e16f4a': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'}}} >Thread-66::DEBUG::2014-05-13 12:49:42,116::task::1185::TaskManager.Task::(prepare) Task=`7ad974cf-008c-412a-b4c6-b01bead3f50e`::finished: {'allTasksStatus': {'6975e26b-13ef-4891-89c8-1d4d34e16f4a': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'}}} >Thread-66::DEBUG::2014-05-13 12:49:42,116::task::595::TaskManager.Task::(_updateState) Task=`7ad974cf-008c-412a-b4c6-b01bead3f50e`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:49:42,116::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:49:42,117::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:49:42,117::task::990::TaskManager.Task::(_decref) Task=`7ad974cf-008c-412a-b4c6-b01bead3f50e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:49:42,375::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:49:42,466::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0432648 s, 23.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:49:42,787::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-344::DEBUG::2014-05-13 12:49:42,823::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:49:42,843::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.00617834 s, 56.3 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:49:44,480::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:49:44,566::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0393436 s, 26.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:49:45,718::sampling::515::root::(_getInterfacesStats) {'network': {'bond4': {'txErrors': '0', 'state': 'down', 'name': 'bond4', 'txDropped': '0', 'txRate': '0.0', 'rxErrors': '0', 'speed': '1000', 'rxRate': '0.0', 'rxDropped': '0'}, 'bond0': {'txErrors': '0', 'state': 'down', 'name': 'bond0', 'txDropped': '0', 'txRate': '0.0', 'rxErrors': '0', 'speed': '1000', 'rxRate': '0.0', 'rxDropped': '0'}, 'bond1': {'txErrors': '0', 'state': 'down', 'name': 'bond1', 'txDropped': '0', 'txRate': '0.0', 'rxErrors': '0', 'speed': '1000', 'rxRate': '0.0', 'rxDropped': '0'}, 'bond2': {'txErrors': '0', 'state': 'down', 'name': 'bond2', 'txDropped': '0', 'txRate': '0.0', 'rxErrors': '0', 'speed': '1000', 'rxRate': '0.0', 'rxDropped': '0'}, 'bond3': {'txErrors': '0', 'state': 'down', 'name': 'bond3', 'txDropped': '0', 'txRate': '0.0', 'rxErrors': '0', 'speed': '1000', 'rxRate': '0.0', 'rxDropped': '0'}, 'lo': {'txErrors': '0', 'state': 'up', 'name': 'lo', 'txDropped': '0', 'txRate': '0.0', 'rxErrors': '0', 'speed': '1000', 'rxRate': '0.0', 'rxDropped': '0'}, ';vdsmdummy;': {'txErrors': '0', 'state': 'down', 'name': ';vdsmdummy;', 'txDropped': '0', 'txRate': '0.0', 'rxErrors': '0', 'speed': '1000', 'rxRate': '0.0', 'rxDropped': '0'}, 'rhevm': {'txErrors': '0', 'state': 'up', 'name': 'rhevm', 'txDropped': '0', 'txRate': '2.7', 'rxErrors': '0', 'speed': '1000', 'rxRate': '93.2', 'rxDropped': '0'}, 'eth1': {'txErrors': '0', 'state': 'down', 'name': 'eth1', 'txDropped': '0', 'txRate': '0.0', 'rxErrors': '0', 'speed': '1000', 'rxRate': '0.0', 'rxDropped': '0'}, 'eth0': {'txErrors': '0', 'state': 'up', 'name': 'eth0', 'txDropped': '0', 'txRate': '2.7', 'rxErrors': '0', 'speed': '1000', 'rxRate': '94.5', 'rxDropped': '0'}}, 'cpuUser': 0.0, 'txRate': 5.3515618766637818, 'statsAge': 1.8274211883544922, 'cpuIdle': 100.0, 'cpuSys': 0.0, 'rxRate': 100.0} >Thread-66::DEBUG::2014-05-13 12:49:45,719::task::595::TaskManager.Task::(_updateState) Task=`702061af-1d70-4e0d-b0f2-31d7f1656756`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:49:45,719::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:49:45,720::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000455442', 'lastCheck': '4.7', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000812008', 'lastCheck': '8.7', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00617834', 'lastCheck': '2.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:49:45,720::task::1185::TaskManager.Task::(prepare) Task=`702061af-1d70-4e0d-b0f2-31d7f1656756`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000455442', 'lastCheck': '4.7', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000812008', 'lastCheck': '8.7', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00617834', 'lastCheck': '2.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:49:45,721::task::595::TaskManager.Task::(_updateState) Task=`702061af-1d70-4e0d-b0f2-31d7f1656756`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:49:45,721::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:49:45,721::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:49:45,721::task::990::TaskManager.Task::(_decref) Task=`702061af-1d70-4e0d-b0f2-31d7f1656756`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:49:45,809::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:49:46,579::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:49:46,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.328641 s, 3.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:49:47,067::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:49:47,075::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000732029 s, 5.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:49:48,827::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:49:48,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:49:49,087::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0549551 s, 18.6 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:49:50,958::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:49:50,959::task::595::TaskManager.Task::(_updateState) Task=`452a992e-3e9a-476d-a6df-f4c1504d4e8f`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:49:50,959::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:49:50,960::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:49:50,960::task::1185::TaskManager.Task::(prepare) Task=`452a992e-3e9a-476d-a6df-f4c1504d4e8f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:49:50,960::task::595::TaskManager.Task::(_updateState) Task=`452a992e-3e9a-476d-a6df-f4c1504d4e8f`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:49:50,960::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:49:50,960::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:49:50,961::task::990::TaskManager.Task::(_decref) Task=`452a992e-3e9a-476d-a6df-f4c1504d4e8f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:49:50,965::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:49:50,965::task::595::TaskManager.Task::(_updateState) Task=`05620feb-3e6a-4340-94c5-544ce0fcdf78`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:49:50,966::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:49:50,966::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d49a38db-fa1b-42fd-925c-3b6c10e0e92a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:49:50,966::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:49:50,967::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:49:50,967::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d49a38db-fa1b-42fd-925c-3b6c10e0e92a`::Granted request >Thread-338::DEBUG::2014-05-13 12:49:50,967::task::827::TaskManager.Task::(resourceAcquired) Task=`05620feb-3e6a-4340-94c5-544ce0fcdf78`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:49:50,967::task::990::TaskManager.Task::(_decref) Task=`05620feb-3e6a-4340-94c5-544ce0fcdf78`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:49:50,968::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:49:50,968::task::1185::TaskManager.Task::(prepare) Task=`05620feb-3e6a-4340-94c5-544ce0fcdf78`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:49:50,969::task::595::TaskManager.Task::(_updateState) Task=`05620feb-3e6a-4340-94c5-544ce0fcdf78`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:49:50,969::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:49:50,969::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:49:50,969::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:49:50,970::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:49:50,970::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:49:50,970::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:49:50,970::task::990::TaskManager.Task::(_decref) Task=`05620feb-3e6a-4340-94c5-544ce0fcdf78`::ref 0 aborting False >Thread-357::DEBUG::2014-05-13 12:49:51,018::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:49:51,029::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000809957 s, 483 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:49:51,104::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:49:51,520::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.370196 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:49:51,844::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:49:52,124::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:49:52,125::task::595::TaskManager.Task::(_updateState) Task=`232cdd04-40c5-4584-8340-aa9ccec13adc`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:49:52,125::logUtils::44::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-66::DEBUG::2014-05-13 12:49:52,125::taskManager::102::TaskManager::(getAllTasksStatuses) Entry. >Thread-66::DEBUG::2014-05-13 12:49:52,125::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 6975e26b-13ef-4891-89c8-1d4d34e16f4a >Thread-66::DEBUG::2014-05-13 12:49:52,125::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'} >Thread-66::DEBUG::2014-05-13 12:49:52,125::taskManager::113::TaskManager::(getAllTasksStatuses) Return: {'6975e26b-13ef-4891-89c8-1d4d34e16f4a': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'}} >Thread-66::INFO::2014-05-13 12:49:52,126::logUtils::47::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {'6975e26b-13ef-4891-89c8-1d4d34e16f4a': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'}}} >Thread-66::DEBUG::2014-05-13 12:49:52,126::task::1185::TaskManager.Task::(prepare) Task=`232cdd04-40c5-4584-8340-aa9ccec13adc`::finished: {'allTasksStatus': {'6975e26b-13ef-4891-89c8-1d4d34e16f4a': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'}}} >Thread-66::DEBUG::2014-05-13 12:49:52,126::task::595::TaskManager.Task::(_updateState) Task=`232cdd04-40c5-4584-8340-aa9ccec13adc`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:49:52,126::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:49:52,126::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:49:52,126::task::990::TaskManager.Task::(_decref) Task=`232cdd04-40c5-4584-8340-aa9ccec13adc`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:49:52,889::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:49:52,910::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.00444899 s, 78.2 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:49:53,533::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:49:53,622::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.043187 s, 23.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:49:54,862::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:49:55,636::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:49:55,778::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0725933 s, 14.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:49:57,091::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:49:57,102::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000671782 s, 6.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:49:57,796::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:49:57,880::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:49:57,890::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.04268 s, 24.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:49:59,903::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:50:00,011::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.060573 s, 16.9 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:50:00,867::sampling::515::root::(_getInterfacesStats) {'network': {'bond4': {'txErrors': '0', 'state': 'down', 'name': 'bond4', 'txDropped': '0', 'txRate': '0.0', 'rxErrors': '0', 'speed': '1000', 'rxRate': '0.0', 'rxDropped': '0'}, 'bond0': {'txErrors': '0', 'state': 'down', 'name': 'bond0', 'txDropped': '0', 'txRate': '0.0', 'rxErrors': '0', 'speed': '1000', 'rxRate': '0.0', 'rxDropped': '0'}, 'bond1': {'txErrors': '0', 'state': 'down', 'name': 'bond1', 'txDropped': '0', 'txRate': '0.0', 'rxErrors': '0', 'speed': '1000', 'rxRate': '0.0', 'rxDropped': '0'}, 'bond2': {'txErrors': '0', 'state': 'down', 'name': 'bond2', 'txDropped': '0', 'txRate': '0.0', 'rxErrors': '0', 'speed': '1000', 'rxRate': '0.0', 'rxDropped': '0'}, 'bond3': {'txErrors': '0', 'state': 'down', 'name': 'bond3', 'txDropped': '0', 'txRate': '0.0', 'rxErrors': '0', 'speed': '1000', 'rxRate': '0.0', 'rxDropped': '0'}, 'lo': {'txErrors': '0', 'state': 'up', 'name': 'lo', 'txDropped': '0', 'txRate': '0.0', 'rxErrors': '0', 'speed': '1000', 'rxRate': '0.0', 'rxDropped': '0'}, ';vdsmdummy;': {'txErrors': '0', 'state': 'down', 'name': ';vdsmdummy;', 'txDropped': '0', 'txRate': '0.0', 'rxErrors': '0', 'speed': '1000', 'rxRate': '0.0', 'rxDropped': '0'}, 'rhevm': {'txErrors': '0', 'state': 'up', 'name': 'rhevm', 'txDropped': '0', 'txRate': '2.7', 'rxErrors': '0', 'speed': '1000', 'rxRate': '93.9', 'rxDropped': '0'}, 'eth1': {'txErrors': '0', 'state': 'down', 'name': 'eth1', 'txDropped': '0', 'txRate': '0.0', 'rxErrors': '0', 'speed': '1000', 'rxRate': '0.0', 'rxDropped': '0'}, 'eth0': {'txErrors': '0', 'state': 'up', 'name': 'eth0', 'txDropped': '0', 'txRate': '2.7', 'rxErrors': '0', 'speed': '1000', 'rxRate': '94.4', 'rxDropped': '0'}}, 'cpuUser': 0.0, 'txRate': 5.3708415869463959, 'statsAge': 0.92748284339904785, 'cpuIdle': 100.0, 'cpuSys': 0.0, 'rxRate': 100.0} >Thread-66::DEBUG::2014-05-13 12:50:00,867::task::595::TaskManager.Task::(_updateState) Task=`c8cd95d4-9219-4ced-9f38-ee32acd2f271`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:50:00,867::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:50:00,868::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000809957', 'lastCheck': '9.8', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000671782', 'lastCheck': '3.8', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00444899', 'lastCheck': '8.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:50:00,868::task::1185::TaskManager.Task::(prepare) Task=`c8cd95d4-9219-4ced-9f38-ee32acd2f271`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000809957', 'lastCheck': '9.8', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000671782', 'lastCheck': '3.8', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00444899', 'lastCheck': '8.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:50:00,868::task::595::TaskManager.Task::(_updateState) Task=`c8cd95d4-9219-4ced-9f38-ee32acd2f271`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:50:00,868::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:50:00,868::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:50:00,869::task::990::TaskManager.Task::(_decref) Task=`c8cd95d4-9219-4ced-9f38-ee32acd2f271`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:50:00,898::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:50:01,011::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:50:01,012::task::595::TaskManager.Task::(_updateState) Task=`3f4d27a2-2dbe-4c2b-825f-aa5a138b867c`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:50:01,012::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:50:01,012::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:50:01,012::task::1185::TaskManager.Task::(prepare) Task=`3f4d27a2-2dbe-4c2b-825f-aa5a138b867c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:50:01,013::task::595::TaskManager.Task::(_updateState) Task=`3f4d27a2-2dbe-4c2b-825f-aa5a138b867c`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:50:01,013::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:50:01,013::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:50:01,013::task::990::TaskManager.Task::(_decref) Task=`3f4d27a2-2dbe-4c2b-825f-aa5a138b867c`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:50:01,017::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:50:01,018::task::595::TaskManager.Task::(_updateState) Task=`b4a14007-7ee3-4540-928d-5c20de9fdf16`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:50:01,018::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:50:01,018::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e4f72e83-bc6d-455a-a82c-723feba40d9a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:50:01,018::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:50:01,019::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:50:01,019::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e4f72e83-bc6d-455a-a82c-723feba40d9a`::Granted request >Thread-338::DEBUG::2014-05-13 12:50:01,019::task::827::TaskManager.Task::(resourceAcquired) Task=`b4a14007-7ee3-4540-928d-5c20de9fdf16`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:50:01,019::task::990::TaskManager.Task::(_decref) Task=`b4a14007-7ee3-4540-928d-5c20de9fdf16`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:50:01,020::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:50:01,020::task::1185::TaskManager.Task::(prepare) Task=`b4a14007-7ee3-4540-928d-5c20de9fdf16`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:50:01,021::task::595::TaskManager.Task::(_updateState) Task=`b4a14007-7ee3-4540-928d-5c20de9fdf16`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:50:01,021::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:50:01,021::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:50:01,021::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:50:01,021::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:50:01,021::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:50:01,022::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:50:01,022::task::990::TaskManager.Task::(_decref) Task=`b4a14007-7ee3-4540-928d-5c20de9fdf16`::ref 0 aborting False >Thread-357::DEBUG::2014-05-13 12:50:01,043::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:50:01,055::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000634001 s, 617 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:50:02,024::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:50:02,131::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0576817 s, 17.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:50:02,134::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:50:02,142::task::595::TaskManager.Task::(_updateState) Task=`f005620a-bc24-486a-8404-5889d194c380`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:50:02,145::logUtils::44::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-66::DEBUG::2014-05-13 12:50:02,146::taskManager::102::TaskManager::(getAllTasksStatuses) Entry. >Thread-66::DEBUG::2014-05-13 12:50:02,147::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 6975e26b-13ef-4891-89c8-1d4d34e16f4a >Thread-66::DEBUG::2014-05-13 12:50:02,148::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'} >Thread-66::DEBUG::2014-05-13 12:50:02,149::taskManager::113::TaskManager::(getAllTasksStatuses) Return: {'6975e26b-13ef-4891-89c8-1d4d34e16f4a': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'}} >Thread-66::INFO::2014-05-13 12:50:02,151::logUtils::47::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {'6975e26b-13ef-4891-89c8-1d4d34e16f4a': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'}}} >Thread-66::DEBUG::2014-05-13 12:50:02,152::task::1185::TaskManager.Task::(prepare) Task=`f005620a-bc24-486a-8404-5889d194c380`::finished: {'allTasksStatus': {'6975e26b-13ef-4891-89c8-1d4d34e16f4a': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'}}} >Thread-66::DEBUG::2014-05-13 12:50:02,153::task::595::TaskManager.Task::(_updateState) Task=`f005620a-bc24-486a-8404-5889d194c380`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:50:02,154::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:50:02,155::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:50:02,156::task::990::TaskManager.Task::(_decref) Task=`f005620a-bc24-486a-8404-5889d194c380`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:50:02,933::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:50:02,958::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.00416042 s, 83.6 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:50:03,916::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:50:04,165::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:50:04,239::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303059 s, 33.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:50:06,252::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:50:06,383::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0607648 s, 16.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:50:06,934::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:50:07,111::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:50:07,122::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000949875 s, 4.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:50:08,400::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:50:08,513::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0695966 s, 14.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:50:09,962::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:50:10,526::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:50:10,646::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0490204 s, 20.9 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:50:11,069::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:50:11,083::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000486124 s, 804 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:50:11,155::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6602059] >Thread-66::DEBUG::2014-05-13 12:50:11,156::task::595::TaskManager.Task::(_updateState) Task=`f072f736-806a-45f6-a8e0-90b746fb7482`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:50:11,156::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:50:11,156::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:50:11,157::task::1185::TaskManager.Task::(prepare) Task=`f072f736-806a-45f6-a8e0-90b746fb7482`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:50:11,157::task::595::TaskManager.Task::(_updateState) Task=`f072f736-806a-45f6-a8e0-90b746fb7482`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:50:11,157::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:50:11,157::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:50:11,157::task::990::TaskManager.Task::(_decref) Task=`f072f736-806a-45f6-a8e0-90b746fb7482`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:50:11,162::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6602059] >Thread-338::DEBUG::2014-05-13 12:50:11,162::task::595::TaskManager.Task::(_updateState) Task=`37eb79e9-4810-42c6-99ad-934ac3e8ac0b`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:50:11,162::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:50:11,163::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`75bb1f17-4a9b-4bfc-805d-496b24ede013`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:50:11,163::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:50:11,163::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:50:11,164::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`75bb1f17-4a9b-4bfc-805d-496b24ede013`::Granted request >Thread-338::DEBUG::2014-05-13 12:50:11,164::task::827::TaskManager.Task::(resourceAcquired) Task=`37eb79e9-4810-42c6-99ad-934ac3e8ac0b`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:50:11,164::task::990::TaskManager.Task::(_decref) Task=`37eb79e9-4810-42c6-99ad-934ac3e8ac0b`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:50:11,165::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:50:11,165::task::1185::TaskManager.Task::(prepare) Task=`37eb79e9-4810-42c6-99ad-934ac3e8ac0b`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:50:11,165::task::595::TaskManager.Task::(_updateState) Task=`37eb79e9-4810-42c6-99ad-934ac3e8ac0b`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:50:11,166::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:50:11,166::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:50:11,166::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:50:11,166::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:50:11,166::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:50:11,167::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:50:11,167::task::990::TaskManager.Task::(_decref) Task=`37eb79e9-4810-42c6-99ad-934ac3e8ac0b`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:50:12,164::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:50:12,197::task::595::TaskManager.Task::(_updateState) Task=`caa63d3a-f11c-4489-a9d3-4922e7564fbc`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:50:12,197::logUtils::44::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-66::DEBUG::2014-05-13 12:50:12,197::taskManager::102::TaskManager::(getAllTasksStatuses) Entry. >Thread-66::DEBUG::2014-05-13 12:50:12,197::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 6975e26b-13ef-4891-89c8-1d4d34e16f4a >Thread-66::DEBUG::2014-05-13 12:50:12,198::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'} >Thread-66::DEBUG::2014-05-13 12:50:12,198::taskManager::113::TaskManager::(getAllTasksStatuses) Return: {'6975e26b-13ef-4891-89c8-1d4d34e16f4a': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'}} >Thread-66::INFO::2014-05-13 12:50:12,198::logUtils::47::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {'6975e26b-13ef-4891-89c8-1d4d34e16f4a': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'}}} >Thread-66::DEBUG::2014-05-13 12:50:12,198::task::1185::TaskManager.Task::(prepare) Task=`caa63d3a-f11c-4489-a9d3-4922e7564fbc`::finished: {'allTasksStatus': {'6975e26b-13ef-4891-89c8-1d4d34e16f4a': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'}}} >Thread-66::DEBUG::2014-05-13 12:50:12,198::task::595::TaskManager.Task::(_updateState) Task=`caa63d3a-f11c-4489-a9d3-4922e7564fbc`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:50:12,198::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:50:12,199::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:50:12,199::task::990::TaskManager.Task::(_decref) Task=`caa63d3a-f11c-4489-a9d3-4922e7564fbc`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:50:12,662::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:50:12,797::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0684692 s, 15.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:50:12,980::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-344::DEBUG::2014-05-13 12:50:13,004::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:50:13,023::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.00428764 s, 81.2 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:50:14,815::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:50:14,950::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0636362 s, 16.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:50:15,998::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:50:16,011::sampling::515::root::(_getInterfacesStats) {'network': {'bond4': {'txErrors': '0', 'state': 'down', 'name': 'bond4', 'txDropped': '0', 'txRate': '0.0', 'rxErrors': '0', 'speed': '1000', 'rxRate': '0.0', 'rxDropped': '0'}, 'bond0': {'txErrors': '0', 'state': 'down', 'name': 'bond0', 'txDropped': '0', 'txRate': '0.0', 'rxErrors': '0', 'speed': '1000', 'rxRate': '0.0', 'rxDropped': '0'}, 'bond1': {'txErrors': '0', 'state': 'down', 'name': 'bond1', 'txDropped': '0', 'txRate': '0.0', 'rxErrors': '0', 'speed': '1000', 'rxRate': '0.0', 'rxDropped': '0'}, 'bond2': {'txErrors': '0', 'state': 'down', 'name': 'bond2', 'txDropped': '0', 'txRate': '0.0', 'rxErrors': '0', 'speed': '1000', 'rxRate': '0.0', 'rxDropped': '0'}, 'bond3': {'txErrors': '0', 'state': 'down', 'name': 'bond3', 'txDropped': '0', 'txRate': '0.0', 'rxErrors': '0', 'speed': '1000', 'rxRate': '0.0', 'rxDropped': '0'}, 'lo': {'txErrors': '0', 'state': 'up', 'name': 'lo', 'txDropped': '0', 'txRate': '0.0', 'rxErrors': '0', 'speed': '1000', 'rxRate': '0.0', 'rxDropped': '0'}, ';vdsmdummy;': {'txErrors': '0', 'state': 'down', 'name': ';vdsmdummy;', 'txDropped': '0', 'txRate': '0.0', 'rxErrors': '0', 'speed': '1000', 'rxRate': '0.0', 'rxDropped': '0'}, 'rhevm': {'txErrors': '0', 'state': 'up', 'name': 'rhevm', 'txDropped': '0', 'txRate': '2.7', 'rxErrors': '0', 'speed': '1000', 'rxRate': '93.9', 'rxDropped': '0'}, 'eth1': {'txErrors': '0', 'state': 'down', 'name': 'eth1', 'txDropped': '0', 'txRate': '0.0', 'rxErrors': '0', 'speed': '1000', 'rxRate': '0.0', 'rxDropped': '0'}, 'eth0': {'txErrors': '0', 'state': 'up', 'name': 'eth0', 'txDropped': '0', 'txRate': '2.7', 'rxErrors': '0', 'speed': '1000', 'rxRate': '94.4', 'rxDropped': '0'}}, 'cpuUser': 0.0, 'txRate': 5.3686726163742495, 'statsAge': 0.01871800422668457, 'cpuIdle': 100.0, 'cpuSys': 0.0, 'rxRate': 100.0} >Thread-66::DEBUG::2014-05-13 12:50:16,012::task::595::TaskManager.Task::(_updateState) Task=`844b897b-4cc7-4b7d-b155-5f1372d30724`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:50:16,012::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:50:16,012::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000486124', 'lastCheck': '4.9', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000949875', 'lastCheck': '8.9', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00428764', 'lastCheck': '3.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:50:16,012::task::1185::TaskManager.Task::(prepare) Task=`844b897b-4cc7-4b7d-b155-5f1372d30724`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000486124', 'lastCheck': '4.9', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000949875', 'lastCheck': '8.9', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00428764', 'lastCheck': '3.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:50:16,013::task::595::TaskManager.Task::(_updateState) Task=`844b897b-4cc7-4b7d-b155-5f1372d30724`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:50:16,013::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:50:16,013::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:50:16,013::task::990::TaskManager.Task::(_decref) Task=`844b897b-4cc7-4b7d-b155-5f1372d30724`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:50:16,966::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 12:50:17,147::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 41830dad-f012-4b4d-b636-863f66d5ca4b >Thread-322::DEBUG::2014-05-13 12:50:17,148::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-322::DEBUG::2014-05-13 12:50:17,151::blockSD::336::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:50:17,455::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.291875 s, 7.0 kB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:50:17,456::misc::287::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.291875 s, 7.0 kB/s'], size: 2048 >Thread-322::DEBUG::2014-05-13 12:50:17,457::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-322::WARNING::2014-05-13 12:50:17,458::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-322::DEBUG::2014-05-13 12:50:17,459::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-322::DEBUG::2014-05-13 12:50:17,461::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:50:17,464::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:50:17,466::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:50:17,468::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:50:17,470::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:50:17,478::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:50:17,548::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.53771 s, 1.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:50:17,755::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:50:17,756::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:50:17,757::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'LOCKPOLICY=', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', 'DESCRIPTION=ISCSI', 'LOGBLKSIZE=512', 'TYPE=ISCSI', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'PHYBLKSIZE=512', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', 'CLASS=Data', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'POOL_DESCRIPTION=DC_2', 'MASTER_VERSION=1', 'ROLE=Master', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', u'POOL_DOMAINS=208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', '_SHA_CKSUM=985bc0a261dab0d9f33927b045f395c62a74e955'] >Thread-322::INFO::2014-05-13 12:50:17,758::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_imageNS already registered >Thread-322::INFO::2014-05-13 12:50:17,758::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_volumeNS already registered >Thread-322::INFO::2014-05-13 12:50:17,758::blockSD::452::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_lvmActivationNS already registered >Thread-322::DEBUG::2014-05-13 12:50:17,759::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 12:50:17,946::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:50:17,951::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:50:17,960::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000695691 s, 5.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:50:19,020::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:50:19,562::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:50:19,655::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0462922 s, 22.1 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:50:21,096::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:50:21,106::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000600936 s, 651 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:50:21,265::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:50:21,266::task::595::TaskManager.Task::(_updateState) Task=`221fd845-50f6-4766-860e-884fc9189c9e`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:50:21,266::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:50:21,267::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:50:21,267::task::1185::TaskManager.Task::(prepare) Task=`221fd845-50f6-4766-860e-884fc9189c9e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:50:21,267::task::595::TaskManager.Task::(_updateState) Task=`221fd845-50f6-4766-860e-884fc9189c9e`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:50:21,267::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:50:21,267::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:50:21,267::task::990::TaskManager.Task::(_decref) Task=`221fd845-50f6-4766-860e-884fc9189c9e`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:50:21,274::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:50:21,274::task::595::TaskManager.Task::(_updateState) Task=`932b388b-3150-4026-a6bd-6fc8d7e7c9f4`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:50:21,274::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:50:21,275::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ddcd5872-3cf6-46d8-8071-8a2b7b5871d3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:50:21,275::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:50:21,275::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:50:21,276::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ddcd5872-3cf6-46d8-8071-8a2b7b5871d3`::Granted request >Thread-338::DEBUG::2014-05-13 12:50:21,276::task::827::TaskManager.Task::(resourceAcquired) Task=`932b388b-3150-4026-a6bd-6fc8d7e7c9f4`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:50:21,276::task::990::TaskManager.Task::(_decref) Task=`932b388b-3150-4026-a6bd-6fc8d7e7c9f4`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:50:21,277::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:50:21,277::task::1185::TaskManager.Task::(prepare) Task=`932b388b-3150-4026-a6bd-6fc8d7e7c9f4`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9595518976', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:50:21,278::task::595::TaskManager.Task::(_updateState) Task=`932b388b-3150-4026-a6bd-6fc8d7e7c9f4`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:50:21,278::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:50:21,278::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:50:21,278::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:50:21,279::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:50:21,279::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:50:21,279::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:50:21,279::task::990::TaskManager.Task::(_decref) Task=`932b388b-3150-4026-a6bd-6fc8d7e7c9f4`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:50:21,669::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:50:21,758::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0426089 s, 24.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:50:22,040::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:50:22,205::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:50:22,205::task::595::TaskManager.Task::(_updateState) Task=`2fafc9ef-6d05-4310-b2d2-ac2997857f5d`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:50:22,206::logUtils::44::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-66::DEBUG::2014-05-13 12:50:22,206::taskManager::102::TaskManager::(getAllTasksStatuses) Entry. >Thread-66::DEBUG::2014-05-13 12:50:22,206::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 6975e26b-13ef-4891-89c8-1d4d34e16f4a >Thread-66::DEBUG::2014-05-13 12:50:22,206::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'} >Thread-66::DEBUG::2014-05-13 12:50:22,207::taskManager::113::TaskManager::(getAllTasksStatuses) Return: {'6975e26b-13ef-4891-89c8-1d4d34e16f4a': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'}} >Thread-66::INFO::2014-05-13 12:50:22,207::logUtils::47::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {'6975e26b-13ef-4891-89c8-1d4d34e16f4a': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'}}} >Thread-66::DEBUG::2014-05-13 12:50:22,207::task::1185::TaskManager.Task::(prepare) Task=`2fafc9ef-6d05-4310-b2d2-ac2997857f5d`::finished: {'allTasksStatus': {'6975e26b-13ef-4891-89c8-1d4d34e16f4a': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'}}} >Thread-66::DEBUG::2014-05-13 12:50:22,207::task::595::TaskManager.Task::(_updateState) Task=`2fafc9ef-6d05-4310-b2d2-ac2997857f5d`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:50:22,207::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:50:22,207::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:50:22,208::task::990::TaskManager.Task::(_decref) Task=`2fafc9ef-6d05-4310-b2d2-ac2997857f5d`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:50:23,047::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:50:23,069::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.00435997 s, 79.8 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:50:23,772::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:50:23,867::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0510743 s, 20.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:50:25,059::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:50:25,211::utils::697::Storage.Misc.excCmd::(watchCmd) SUCCESS: <err> = []; <rc> = 0 >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:50:25,212::volume::1111::Storage.Volume::(qemuConvert) (qemuConvert): COPY /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/572b0496-5eab-41cd-a976-4514f111898e/images/ac82bfa3-8cd5-4643-b36a-5ad28040fe0a/b4c44053-1991-4c45-aea6-2bcb369e4b65 to /rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/images/f35efc5c-cd67-4de9-b1c6-9e08bafcae46/c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a DONE >6975e26b-13ef-4891-89c8-1d4d34e16f4a::INFO::2014-05-13 12:50:25,224::volume::646::Storage.Volume::(setLegality) sdUUID=572b0496-5eab-41cd-a976-4514f111898e imgUUID=f35efc5c-cd67-4de9-b1c6-9e08bafcae46 volUUID = c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a legality = LEGAL >6975e26b-13ef-4891-89c8-1d4d34e16f4a::INFO::2014-05-13 12:50:25,234::image::780::Storage.Image::(copyCollapsed) Finished copying 572b0496-5eab-41cd-a976-4514f111898e:b4c44053-1991-4c45-aea6-2bcb369e4b65 -> 572b0496-5eab-41cd-a976-4514f111898e:c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:50:25,235::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource '572b0496-5eab-41cd-a976-4514f111898e_imageNS.f35efc5c-cd67-4de9-b1c6-9e08bafcae46' >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:50:25,235::resourceManager::635::ResourceManager::(releaseResource) Released resource '572b0496-5eab-41cd-a976-4514f111898e_imageNS.f35efc5c-cd67-4de9-b1c6-9e08bafcae46' (0 active users) >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:50:25,235::resourceManager::641::ResourceManager::(releaseResource) Resource '572b0496-5eab-41cd-a976-4514f111898e_imageNS.f35efc5c-cd67-4de9-b1c6-9e08bafcae46' is free, finding out if anyone is waiting for it. >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:50:25,235::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource '572b0496-5eab-41cd-a976-4514f111898e_imageNS.f35efc5c-cd67-4de9-b1c6-9e08bafcae46', Clearing records. >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:50:25,236::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource '572b0496-5eab-41cd-a976-4514f111898e_imageNS.ac82bfa3-8cd5-4643-b36a-5ad28040fe0a' >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:50:25,236::resourceManager::635::ResourceManager::(releaseResource) Released resource '572b0496-5eab-41cd-a976-4514f111898e_imageNS.ac82bfa3-8cd5-4643-b36a-5ad28040fe0a' (0 active users) >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:50:25,236::resourceManager::641::ResourceManager::(releaseResource) Resource '572b0496-5eab-41cd-a976-4514f111898e_imageNS.ac82bfa3-8cd5-4643-b36a-5ad28040fe0a' is free, finding out if anyone is waiting for it. >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:50:25,237::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource '572b0496-5eab-41cd-a976-4514f111898e_volumeNS.b4c44053-1991-4c45-aea6-2bcb369e4b65' >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:50:25,237::resourceManager::635::ResourceManager::(releaseResource) Released resource '572b0496-5eab-41cd-a976-4514f111898e_volumeNS.b4c44053-1991-4c45-aea6-2bcb369e4b65' (0 active users) >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:50:25,237::resourceManager::641::ResourceManager::(releaseResource) Resource '572b0496-5eab-41cd-a976-4514f111898e_volumeNS.b4c44053-1991-4c45-aea6-2bcb369e4b65' is free, finding out if anyone is waiting for it. >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:50:25,237::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource '572b0496-5eab-41cd-a976-4514f111898e_volumeNS.b4c44053-1991-4c45-aea6-2bcb369e4b65', Clearing records. >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:50:25,237::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource '572b0496-5eab-41cd-a976-4514f111898e_imageNS.ac82bfa3-8cd5-4643-b36a-5ad28040fe0a', Clearing records. >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:50:25,238::task::595::TaskManager.Task::(_updateState) Task=`6975e26b-13ef-4891-89c8-1d4d34e16f4a`::moving from state running -> state finished >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:50:25,238::task::752::TaskManager.Task::(_save) Task=`6975e26b-13ef-4891-89c8-1d4d34e16f4a`::_save: orig /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/6975e26b-13ef-4891-89c8-1d4d34e16f4a temp /rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/master/tasks/6975e26b-13ef-4891-89c8-1d4d34e16f4a.temp >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:50:25,249::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.572b0496-5eab-41cd-a976-4514f111898e': < ResourceRef 'Storage.572b0496-5eab-41cd-a976-4514f111898e', isValid: 'True' obj: 'None'>} >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:50:25,250::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:50:25,250::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:50:25,250::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' (0 active users) >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:50:25,250::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' is free, finding out if anyone is waiting for it. >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:50:25,250::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e', Clearing records. >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:50:25,251::task::913::TaskManager.Task::(_runJobs) Task=`6975e26b-13ef-4891-89c8-1d4d34e16f4a`::Task.run: exit - success: result {'uuid': 'c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a'} >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:50:25,251::task::990::TaskManager.Task::(_decref) Task=`6975e26b-13ef-4891-89c8-1d4d34e16f4a`::ref 0 aborting False >6975e26b-13ef-4891-89c8-1d4d34e16f4a::DEBUG::2014-05-13 12:50:25,251::threadPool::57::Misc.ThreadPool::(setRunningTask) Number of running tasks: 0 >Dummy-337::DEBUG::2014-05-13 12:50:25,881::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:50:25,960::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361437 s, 28.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:50:27,972::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:50:27,979::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 12:50:27,990::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000558295 s, 7.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:50:28,073::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364573 s, 28.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:50:28,077::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:50:30,088::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:50:30,171::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037781 s, 27.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:50:31,102::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-357::DEBUG::2014-05-13 12:50:31,118::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:50:31,127::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000505444 s, 774 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:50:31,149::task::595::TaskManager.Task::(_updateState) Task=`54b91d7c-2a8e-4844-935e-124b6ef6d076`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:50:31,149::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:50:31,150::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000505444', 'lastCheck': '0.0', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000558295', 'lastCheck': '3.2', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00435997', 'lastCheck': '8.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:50:31,150::task::1185::TaskManager.Task::(prepare) Task=`54b91d7c-2a8e-4844-935e-124b6ef6d076`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000505444', 'lastCheck': '0.0', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000558295', 'lastCheck': '3.2', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00435997', 'lastCheck': '8.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:50:31,150::task::595::TaskManager.Task::(_updateState) Task=`54b91d7c-2a8e-4844-935e-124b6ef6d076`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:50:31,150::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:50:31,150::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:50:31,151::task::990::TaskManager.Task::(_decref) Task=`54b91d7c-2a8e-4844-935e-124b6ef6d076`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:50:31,318::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5eff0b16] >Thread-66::DEBUG::2014-05-13 12:50:31,319::task::595::TaskManager.Task::(_updateState) Task=`27aa7adc-543f-49de-b701-62283facd942`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:50:31,319::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:50:31,319::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:50:31,319::task::1185::TaskManager.Task::(prepare) Task=`27aa7adc-543f-49de-b701-62283facd942`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:50:31,319::task::595::TaskManager.Task::(_updateState) Task=`27aa7adc-543f-49de-b701-62283facd942`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:50:31,320::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:50:31,320::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:50:31,320::task::990::TaskManager.Task::(_decref) Task=`27aa7adc-543f-49de-b701-62283facd942`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:50:31,324::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [5eff0b16] >Thread-338::DEBUG::2014-05-13 12:50:31,325::task::595::TaskManager.Task::(_updateState) Task=`74c87e44-29b5-421d-abf9-4d24e11b280e`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:50:31,325::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:50:31,325::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c871e28b-1add-4211-ad78-c78455379a25`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:50:31,325::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:50:31,326::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:50:31,326::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`c871e28b-1add-4211-ad78-c78455379a25`::Granted request >Thread-338::DEBUG::2014-05-13 12:50:31,326::task::827::TaskManager.Task::(resourceAcquired) Task=`74c87e44-29b5-421d-abf9-4d24e11b280e`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:50:31,326::task::990::TaskManager.Task::(_decref) Task=`74c87e44-29b5-421d-abf9-4d24e11b280e`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:50:31,327::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9598664704', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:50:31,327::task::1185::TaskManager.Task::(prepare) Task=`74c87e44-29b5-421d-abf9-4d24e11b280e`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9598664704', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:50:31,328::task::595::TaskManager.Task::(_updateState) Task=`74c87e44-29b5-421d-abf9-4d24e11b280e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:50:31,328::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:50:31,328::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:50:31,328::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:50:31,329::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:50:31,329::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:50:31,329::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:50:31,329::task::990::TaskManager.Task::(_decref) Task=`74c87e44-29b5-421d-abf9-4d24e11b280e`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:50:32,184::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 12:50:32,214::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [206acf2a] >Thread-66::DEBUG::2014-05-13 12:50:32,216::task::595::TaskManager.Task::(_updateState) Task=`7dfec529-44c6-4f88-8fb0-b99c3b0295a6`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:50:32,217::logUtils::44::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-66::DEBUG::2014-05-13 12:50:32,218::taskManager::102::TaskManager::(getAllTasksStatuses) Entry. >Thread-66::DEBUG::2014-05-13 12:50:32,218::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 6975e26b-13ef-4891-89c8-1d4d34e16f4a >Thread-66::DEBUG::2014-05-13 12:50:32,219::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'} >Thread-66::DEBUG::2014-05-13 12:50:32,220::taskManager::113::TaskManager::(getAllTasksStatuses) Return: {'6975e26b-13ef-4891-89c8-1d4d34e16f4a': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'}} >Thread-66::INFO::2014-05-13 12:50:32,221::logUtils::47::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {'6975e26b-13ef-4891-89c8-1d4d34e16f4a': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'}}} >Thread-66::DEBUG::2014-05-13 12:50:32,222::task::1185::TaskManager.Task::(prepare) Task=`7dfec529-44c6-4f88-8fb0-b99c3b0295a6`::finished: {'allTasksStatus': {'6975e26b-13ef-4891-89c8-1d4d34e16f4a': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '6975e26b-13ef-4891-89c8-1d4d34e16f4a'}}} >Thread-66::DEBUG::2014-05-13 12:50:32,223::task::595::TaskManager.Task::(_updateState) Task=`7dfec529-44c6-4f88-8fb0-b99c3b0295a6`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:50:32,225::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:50:32,225::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:50:32,227::task::990::TaskManager.Task::(_decref) Task=`7dfec529-44c6-4f88-8fb0-b99c3b0295a6`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:50:32,280::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0523222 s, 19.6 MB/s\n'; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:50:32,296::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6e0ac8a1] >Thread-338::DEBUG::2014-05-13 12:50:32,296::task::595::TaskManager.Task::(_updateState) Task=`dada5d68-e665-4f90-bfc4-16524795a4b0`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:50:32,297::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeInfo(sdUUID='572b0496-5eab-41cd-a976-4514f111898e', spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', imgUUID='f35efc5c-cd67-4de9-b1c6-9e08bafcae46', volUUID='c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a', options=None) >Thread-338::DEBUG::2014-05-13 12:50:32,297::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.572b0496-5eab-41cd-a976-4514f111898e`ReqID=`80001100-741e-4ffd-bc46-967aed109148`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '3089' at 'getVolumeInfo' >Thread-338::DEBUG::2014-05-13 12:50:32,297::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:50:32,297::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:50:32,298::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.572b0496-5eab-41cd-a976-4514f111898e`ReqID=`80001100-741e-4ffd-bc46-967aed109148`::Granted request >Thread-338::DEBUG::2014-05-13 12:50:32,298::task::827::TaskManager.Task::(resourceAcquired) Task=`dada5d68-e665-4f90-bfc4-16524795a4b0`::_resourcesAcquired: Storage.572b0496-5eab-41cd-a976-4514f111898e (shared) >Thread-338::DEBUG::2014-05-13 12:50:32,298::task::990::TaskManager.Task::(_decref) Task=`dada5d68-e665-4f90-bfc4-16524795a4b0`::ref 1 aborting False >Thread-338::DEBUG::2014-05-13 12:50:32,300::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a >Thread-338::INFO::2014-05-13 12:50:32,302::volume::879::Storage.Volume::(getInfo) Info request: sdUUID=572b0496-5eab-41cd-a976-4514f111898e imgUUID=f35efc5c-cd67-4de9-b1c6-9e08bafcae46 volUUID = c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a >Thread-338::INFO::2014-05-13 12:50:32,327::volume::909::Storage.Volume::(getInfo) 572b0496-5eab-41cd-a976-4514f111898e/f35efc5c-cd67-4de9-b1c6-9e08bafcae46/c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a info is {'status': 'OK', 'domain': '572b0496-5eab-41cd-a976-4514f111898e', 'voltype': 'SHARED', 'description': 'Active VM', 'parent': '00000000-0000-0000-0000-000000000000', 'format': 'RAW', 'image': 'f35efc5c-cd67-4de9-b1c6-9e08bafcae46', 'ctime': '1399974569', 'disktype': '2', 'legality': 'LEGAL', 'mtime': '1399974569', 'apparentsize': '5368709120', 'children': [], 'pool': '', 'capacity': '5368709120', 'uuid': 'c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a', 'truesize': '24576', 'type': 'SPARSE'} >Thread-338::INFO::2014-05-13 12:50:32,327::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeInfo, Return response: {'info': {'status': 'OK', 'domain': '572b0496-5eab-41cd-a976-4514f111898e', 'voltype': 'SHARED', 'description': 'Active VM', 'parent': '00000000-0000-0000-0000-000000000000', 'format': 'RAW', 'image': 'f35efc5c-cd67-4de9-b1c6-9e08bafcae46', 'ctime': '1399974569', 'disktype': '2', 'legality': 'LEGAL', 'mtime': '1399974569', 'apparentsize': '5368709120', 'children': [], 'pool': '', 'capacity': '5368709120', 'uuid': 'c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a', 'truesize': '24576', 'type': 'SPARSE'}} >Thread-338::DEBUG::2014-05-13 12:50:32,327::task::1185::TaskManager.Task::(prepare) Task=`dada5d68-e665-4f90-bfc4-16524795a4b0`::finished: {'info': {'status': 'OK', 'domain': '572b0496-5eab-41cd-a976-4514f111898e', 'voltype': 'SHARED', 'description': 'Active VM', 'parent': '00000000-0000-0000-0000-000000000000', 'format': 'RAW', 'image': 'f35efc5c-cd67-4de9-b1c6-9e08bafcae46', 'ctime': '1399974569', 'disktype': '2', 'legality': 'LEGAL', 'mtime': '1399974569', 'apparentsize': '5368709120', 'children': [], 'pool': '', 'capacity': '5368709120', 'uuid': 'c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a', 'truesize': '24576', 'type': 'SPARSE'}} >Thread-338::DEBUG::2014-05-13 12:50:32,327::task::595::TaskManager.Task::(_updateState) Task=`dada5d68-e665-4f90-bfc4-16524795a4b0`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:50:32,328::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.572b0496-5eab-41cd-a976-4514f111898e': < ResourceRef 'Storage.572b0496-5eab-41cd-a976-4514f111898e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:50:32,328::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:50:32,328::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' >Thread-338::DEBUG::2014-05-13 12:50:32,328::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:50:32,328::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:50:32,329::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.572b0496-5eab-41cd-a976-4514f111898e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:50:32,329::task::990::TaskManager.Task::(_decref) Task=`dada5d68-e665-4f90-bfc4-16524795a4b0`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:50:32,421::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:50:32,421::task::595::TaskManager.Task::(_updateState) Task=`0aac57e2-4c5c-4c67-ad79-c63c192f28e5`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:50:32,422::logUtils::44::dispatcher::(wrapper) Run and protect: clearTask(taskID='6975e26b-13ef-4891-89c8-1d4d34e16f4a', spUUID=None, options=None) >Thread-66::DEBUG::2014-05-13 12:50:32,422::taskManager::161::TaskManager::(clearTask) Entry. taskID: 6975e26b-13ef-4891-89c8-1d4d34e16f4a >Thread-66::DEBUG::2014-05-13 12:50:32,423::taskManager::166::TaskManager::(clearTask) Return. >Thread-66::INFO::2014-05-13 12:50:32,423::logUtils::47::dispatcher::(wrapper) Run and protect: clearTask, Return response: None >Thread-66::DEBUG::2014-05-13 12:50:32,423::task::1185::TaskManager.Task::(prepare) Task=`0aac57e2-4c5c-4c67-ad79-c63c192f28e5`::finished: None >Thread-66::DEBUG::2014-05-13 12:50:32,423::task::595::TaskManager.Task::(_updateState) Task=`0aac57e2-4c5c-4c67-ad79-c63c192f28e5`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:50:32,423::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:50:32,424::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:50:32,424::task::990::TaskManager.Task::(_decref) Task=`0aac57e2-4c5c-4c67-ad79-c63c192f28e5`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:50:33,120::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:50:33,129::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000411435 s, 846 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:50:34,120::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:50:34,293::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:50:34,372::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0358226 s, 28.6 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:50:36,385::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:50:36,468::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384517 s, 26.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:50:37,139::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:50:37,999::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:50:38,007::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000773281 s, 5.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:50:38,481::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:50:38,563::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379827 s, 27.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:50:40,168::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:50:40,576::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:50:40,665::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0450602 s, 22.7 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:50:41,138::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:50:41,148::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.00060443 s, 647 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:50:41,363::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:50:41,363::task::595::TaskManager.Task::(_updateState) Task=`cdd36791-fc91-42b5-a0f4-a40bfa1a7d77`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:50:41,363::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:50:41,364::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:50:41,364::task::1185::TaskManager.Task::(prepare) Task=`cdd36791-fc91-42b5-a0f4-a40bfa1a7d77`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:50:41,364::task::595::TaskManager.Task::(_updateState) Task=`cdd36791-fc91-42b5-a0f4-a40bfa1a7d77`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:50:41,364::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:50:41,365::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:50:41,365::task::990::TaskManager.Task::(_decref) Task=`cdd36791-fc91-42b5-a0f4-a40bfa1a7d77`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:50:41,370::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:50:41,370::task::595::TaskManager.Task::(_updateState) Task=`f3e3080f-a34e-4f66-a3a1-1199b52b2976`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:50:41,370::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:50:41,371::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`03661263-2909-4449-8608-232e8ec5cdfa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:50:41,371::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:50:41,371::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:50:41,371::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`03661263-2909-4449-8608-232e8ec5cdfa`::Granted request >Thread-338::DEBUG::2014-05-13 12:50:41,373::task::827::TaskManager.Task::(resourceAcquired) Task=`f3e3080f-a34e-4f66-a3a1-1199b52b2976`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:50:41,373::task::990::TaskManager.Task::(_decref) Task=`f3e3080f-a34e-4f66-a3a1-1199b52b2976`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:50:41,374::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9598664704', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:50:41,374::task::1185::TaskManager.Task::(prepare) Task=`f3e3080f-a34e-4f66-a3a1-1199b52b2976`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9598664704', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:50:41,374::task::595::TaskManager.Task::(_updateState) Task=`f3e3080f-a34e-4f66-a3a1-1199b52b2976`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:50:41,374::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:50:41,375::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:50:41,375::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:50:41,375::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:50:41,375::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:50:41,375::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:50:41,376::task::990::TaskManager.Task::(_decref) Task=`f3e3080f-a34e-4f66-a3a1-1199b52b2976`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:50:42,678::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:50:42,760::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.038646 s, 26.5 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:50:43,147::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:50:43,156::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000427339 s, 814 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:50:43,187::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:50:44,773::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:50:44,853::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381456 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:50:46,214::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:50:46,288::task::595::TaskManager.Task::(_updateState) Task=`f64d80d2-d435-452b-bb14-bc8c3c27c8f1`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:50:46,288::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:50:46,289::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00060443', 'lastCheck': '5.1', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000773281', 'lastCheck': '8.3', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000427339', 'lastCheck': '3.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:50:46,289::task::1185::TaskManager.Task::(prepare) Task=`f64d80d2-d435-452b-bb14-bc8c3c27c8f1`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00060443', 'lastCheck': '5.1', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000773281', 'lastCheck': '8.3', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000427339', 'lastCheck': '3.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:50:46,289::task::595::TaskManager.Task::(_updateState) Task=`f64d80d2-d435-452b-bb14-bc8c3c27c8f1`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:50:46,289::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:50:46,289::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:50:46,290::task::990::TaskManager.Task::(_decref) Task=`f64d80d2-d435-452b-bb14-bc8c3c27c8f1`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:50:46,866::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:50:46,944::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0336978 s, 30.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:50:48,016::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:50:48,025::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00185466 s, 2.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:50:48,957::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:50:49,236::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:50:49,250::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357538 s, 28.6 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:50:51,160::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:50:51,169::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000577383 s, 677 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:50:51,263::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:50:51,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387586 s, 26.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:50:51,411::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3256e43f] >Thread-66::DEBUG::2014-05-13 12:50:51,411::task::595::TaskManager.Task::(_updateState) Task=`5e08bc09-4fb3-45a7-9822-f07588486b12`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:50:51,411::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:50:51,412::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:50:51,412::task::1185::TaskManager.Task::(prepare) Task=`5e08bc09-4fb3-45a7-9822-f07588486b12`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:50:51,412::task::595::TaskManager.Task::(_updateState) Task=`5e08bc09-4fb3-45a7-9822-f07588486b12`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:50:51,412::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:50:51,413::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:50:51,413::task::990::TaskManager.Task::(_decref) Task=`5e08bc09-4fb3-45a7-9822-f07588486b12`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:50:51,417::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3256e43f] >Thread-338::DEBUG::2014-05-13 12:50:51,418::task::595::TaskManager.Task::(_updateState) Task=`bbc234b9-6fee-4556-a65e-2c5cb38c1205`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:50:51,418::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:50:51,418::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0aff2a11-7b37-4f4e-a81a-48bca29bbfe0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:50:51,419::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:50:51,419::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:50:51,419::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0aff2a11-7b37-4f4e-a81a-48bca29bbfe0`::Granted request >Thread-338::DEBUG::2014-05-13 12:50:51,419::task::827::TaskManager.Task::(resourceAcquired) Task=`bbc234b9-6fee-4556-a65e-2c5cb38c1205`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:50:51,420::task::990::TaskManager.Task::(_decref) Task=`bbc234b9-6fee-4556-a65e-2c5cb38c1205`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:50:51,421::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9598664704', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:50:51,421::task::1185::TaskManager.Task::(prepare) Task=`bbc234b9-6fee-4556-a65e-2c5cb38c1205`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9598664704', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:50:51,421::task::595::TaskManager.Task::(_updateState) Task=`bbc234b9-6fee-4556-a65e-2c5cb38c1205`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:50:51,421::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:50:51,421::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:50:51,422::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:50:51,422::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:50:51,422::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:50:51,422::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:50:51,423::task::990::TaskManager.Task::(_decref) Task=`bbc234b9-6fee-4556-a65e-2c5cb38c1205`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:50:52,255::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-344::DEBUG::2014-05-13 12:50:53,166::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:50:53,175::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000496602 s, 701 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:50:53,357::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:50:53,810::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.410844 s, 2.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:50:55,274::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:50:55,823::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:50:55,897::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307585 s, 33.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:50:57,910::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:50:57,986::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322033 s, 31.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:50:58,034::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:50:58,043::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000647835 s, 6.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:50:58,296::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:50:59,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:00,078::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0369047 s, 27.7 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:51:01,181::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:51:01,190::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.00054054 s, 723 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:51:01,316::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:51:01,424::task::595::TaskManager.Task::(_updateState) Task=`f810fe3a-7971-4ee7-bd0a-f2dd6ad92cfc`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:51:01,424::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:51:01,424::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00054054', 'lastCheck': '0.2', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000647835', 'lastCheck': '3.4', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000496602', 'lastCheck': '8.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:51:01,424::task::1185::TaskManager.Task::(prepare) Task=`f810fe3a-7971-4ee7-bd0a-f2dd6ad92cfc`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00054054', 'lastCheck': '0.2', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000647835', 'lastCheck': '3.4', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000496602', 'lastCheck': '8.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:51:01,425::task::595::TaskManager.Task::(_updateState) Task=`f810fe3a-7971-4ee7-bd0a-f2dd6ad92cfc`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:51:01,425::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:51:01,425::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:51:01,425::task::990::TaskManager.Task::(_decref) Task=`f810fe3a-7971-4ee7-bd0a-f2dd6ad92cfc`::ref 0 aborting False >Thread-13::DEBUG::2014-05-13 12:51:01,461::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-13::DEBUG::2014-05-13 12:51:01,461::task::595::TaskManager.Task::(_updateState) Task=`9554c2c5-2714-470f-9ecb-71f9097a98dc`::moving from state init -> state preparing >Thread-13::INFO::2014-05-13 12:51:01,461::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-13::INFO::2014-05-13 12:51:01,462::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:51:01,462::task::1185::TaskManager.Task::(prepare) Task=`9554c2c5-2714-470f-9ecb-71f9097a98dc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-13::DEBUG::2014-05-13 12:51:01,462::task::595::TaskManager.Task::(_updateState) Task=`9554c2c5-2714-470f-9ecb-71f9097a98dc`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-05-13 12:51:01,462::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-05-13 12:51:01,462::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-05-13 12:51:01,463::task::990::TaskManager.Task::(_decref) Task=`9554c2c5-2714-470f-9ecb-71f9097a98dc`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:51:01,467::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:51:01,468::task::595::TaskManager.Task::(_updateState) Task=`1b2614a2-924f-46cb-8797-60cc3fb4d56c`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:51:01,468::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:51:01,468::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b262cf8e-ce89-4ea9-bf38-c643ff9aea98`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:51:01,469::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:51:01,469::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:51:01,469::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b262cf8e-ce89-4ea9-bf38-c643ff9aea98`::Granted request >Thread-338::DEBUG::2014-05-13 12:51:01,469::task::827::TaskManager.Task::(resourceAcquired) Task=`1b2614a2-924f-46cb-8797-60cc3fb4d56c`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:51:01,470::task::990::TaskManager.Task::(_decref) Task=`1b2614a2-924f-46cb-8797-60cc3fb4d56c`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:51:01,471::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9598664704', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:51:01,471::task::1185::TaskManager.Task::(prepare) Task=`1b2614a2-924f-46cb-8797-60cc3fb4d56c`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9598664704', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:51:01,471::task::595::TaskManager.Task::(_updateState) Task=`1b2614a2-924f-46cb-8797-60cc3fb4d56c`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:51:01,471::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:51:01,471::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:51:01,472::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:51:01,472::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:51:01,472::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:51:01,472::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:51:01,472::task::990::TaskManager.Task::(_decref) Task=`1b2614a2-924f-46cb-8797-60cc3fb4d56c`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:51:02,091::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:02,185::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0508208 s, 20.1 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:51:03,186::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:51:03,195::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000445549 s, 781 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:51:04,197::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:04,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379096 s, 27.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:51:04,334::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:51:06,291::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:06,375::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0413959 s, 24.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:51:07,351::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:51:08,050::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:51:08,059::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000817271 s, 5.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:51:08,388::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:08,482::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0519096 s, 19.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:51:10,382::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:51:10,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:10,576::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323785 s, 31.6 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:51:11,193::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 208946c2-5db1-4ef4-80b1-9aeb37f2905e >Thread-357::DEBUG::2014-05-13 12:51:11,202::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e >Thread-357::DEBUG::2014-05-13 12:51:11,204::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-357::DEBUG::2014-05-13 12:51:11,211::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=ISO_DOMAIN', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', 'REMOTE_PATH=dhcp163-66.scl.lab.tlv.redhat.com:/var/lib/exports/iso-20140512085215', 'ROLE=Regular', 'SDUUID=208946c2-5db1-4ef4-80b1-9aeb37f2905e', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=0be9ca2275e0b4f627ac4aeccb1d1475df0d42f5'] >Thread-357::DEBUG::2014-05-13 12:51:11,213::fileSD::609::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-357::INFO::2014-05-13 12:51:11,214::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 208946c2-5db1-4ef4-80b1-9aeb37f2905e_imageNS already registered >Thread-357::INFO::2014-05-13 12:51:11,214::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 208946c2-5db1-4ef4-80b1-9aeb37f2905e_volumeNS already registered >Thread-357::DEBUG::2014-05-13 12:51:11,223::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:51:11,232::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000429805 s, 910 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:51:11,527::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6019f20] >Thread-66::DEBUG::2014-05-13 12:51:11,528::task::595::TaskManager.Task::(_updateState) Task=`8a378727-11c8-4cf8-bab8-2dd72bc47b89`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:51:11,528::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:51:11,528::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:51:11,529::task::1185::TaskManager.Task::(prepare) Task=`8a378727-11c8-4cf8-bab8-2dd72bc47b89`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:51:11,529::task::595::TaskManager.Task::(_updateState) Task=`8a378727-11c8-4cf8-bab8-2dd72bc47b89`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:51:11,529::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:51:11,529::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:51:11,529::task::990::TaskManager.Task::(_decref) Task=`8a378727-11c8-4cf8-bab8-2dd72bc47b89`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:51:11,540::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6019f20] >Thread-338::DEBUG::2014-05-13 12:51:11,541::task::595::TaskManager.Task::(_updateState) Task=`01784769-a91e-4cf1-ba25-e019c07804a1`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:51:11,541::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:51:11,541::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`795367ba-0434-4c8f-8314-0e0b7a8d8d59`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:51:11,542::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:51:11,542::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:51:11,542::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`795367ba-0434-4c8f-8314-0e0b7a8d8d59`::Granted request >Thread-338::DEBUG::2014-05-13 12:51:11,543::task::827::TaskManager.Task::(resourceAcquired) Task=`01784769-a91e-4cf1-ba25-e019c07804a1`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:51:11,543::task::990::TaskManager.Task::(_decref) Task=`01784769-a91e-4cf1-ba25-e019c07804a1`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:51:11,544::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9598664704', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:51:11,544::task::1185::TaskManager.Task::(prepare) Task=`01784769-a91e-4cf1-ba25-e019c07804a1`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9598664704', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:51:11,544::task::595::TaskManager.Task::(_updateState) Task=`01784769-a91e-4cf1-ba25-e019c07804a1`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:51:11,544::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:51:11,544::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:51:11,545::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:51:11,545::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:51:11,545::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:51:11,545::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:51:11,545::task::990::TaskManager.Task::(_decref) Task=`01784769-a91e-4cf1-ba25-e019c07804a1`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:51:12,588::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:12,669::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0384105 s, 26.7 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:51:13,205::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:51:13,214::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.00035183 s, 989 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:51:13,404::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:51:14,682::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:14,763::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.038668 s, 26.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:51:16,423::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:51:16,577::task::595::TaskManager.Task::(_updateState) Task=`fcfaece4-307f-4375-94de-0ef4cdd1d736`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:51:16,617::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:51:16,617::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000429805', 'lastCheck': '5.4', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000817271', 'lastCheck': '8.6', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00035183', 'lastCheck': '3.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:51:16,617::task::1185::TaskManager.Task::(prepare) Task=`fcfaece4-307f-4375-94de-0ef4cdd1d736`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000429805', 'lastCheck': '5.4', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000817271', 'lastCheck': '8.6', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00035183', 'lastCheck': '3.4', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:51:16,617::task::595::TaskManager.Task::(_updateState) Task=`fcfaece4-307f-4375-94de-0ef4cdd1d736`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:51:16,617::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:51:16,618::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:51:16,618::task::990::TaskManager.Task::(_decref) Task=`fcfaece4-307f-4375-94de-0ef4cdd1d736`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:51:16,776::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:16,849::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0301816 s, 33.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:51:18,067::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:51:18,076::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000612627 s, 6.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:51:18,862::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:18,938::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360469 s, 28.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:51:19,441::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:51:20,946::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:21,025::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357 s, 28.7 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:51:21,244::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:51:21,254::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000503241 s, 777 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:51:21,581::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [45b9a91f] >Thread-66::DEBUG::2014-05-13 12:51:21,618::task::595::TaskManager.Task::(_updateState) Task=`ca0af9c3-c4d0-4075-851d-6003c53decb1`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:51:21,619::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:51:21,619::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:51:21,619::task::1185::TaskManager.Task::(prepare) Task=`ca0af9c3-c4d0-4075-851d-6003c53decb1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:51:21,619::task::595::TaskManager.Task::(_updateState) Task=`ca0af9c3-c4d0-4075-851d-6003c53decb1`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:51:21,619::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:51:21,620::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:51:21,620::task::990::TaskManager.Task::(_decref) Task=`ca0af9c3-c4d0-4075-851d-6003c53decb1`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:51:21,626::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [45b9a91f] >Thread-338::DEBUG::2014-05-13 12:51:21,626::task::595::TaskManager.Task::(_updateState) Task=`f880ce9c-976e-4085-9118-2436e86d4766`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:51:21,626::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:51:21,627::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e990480e-f0bb-4b4a-923a-171e2d5ef028`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:51:21,627::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:51:21,627::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:51:21,628::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e990480e-f0bb-4b4a-923a-171e2d5ef028`::Granted request >Thread-338::DEBUG::2014-05-13 12:51:21,628::task::827::TaskManager.Task::(resourceAcquired) Task=`f880ce9c-976e-4085-9118-2436e86d4766`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:51:21,628::task::990::TaskManager.Task::(_decref) Task=`f880ce9c-976e-4085-9118-2436e86d4766`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:51:21,629::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9598664704', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:51:21,629::task::1185::TaskManager.Task::(prepare) Task=`f880ce9c-976e-4085-9118-2436e86d4766`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9598664704', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:51:21,630::task::595::TaskManager.Task::(_updateState) Task=`f880ce9c-976e-4085-9118-2436e86d4766`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:51:21,630::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:51:21,630::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:51:21,630::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:51:21,630::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:51:21,631::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:51:21,631::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:51:21,631::task::990::TaskManager.Task::(_decref) Task=`f880ce9c-976e-4085-9118-2436e86d4766`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:51:22,464::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:51:23,038::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:23,127::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0447477 s, 22.9 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:51:23,225::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:51:23,235::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000416569 s, 835 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:51:25,141::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:25,216::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0326998 s, 31.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:51:25,483::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:51:27,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:27,309::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0372154 s, 27.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:51:28,084::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:51:28,092::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000657313 s, 6.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:51:28,502::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:51:29,322::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:29,638::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.14666 s, 7.0 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:51:31,264::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:51:31,274::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000490838 s, 797 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:51:31,522::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:51:31,651::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 12:51:31,671::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1c80df6] >Thread-66::DEBUG::2014-05-13 12:51:31,673::task::595::TaskManager.Task::(_updateState) Task=`4829c1bd-5074-4730-9aa9-b39efc9c92d2`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:51:31,675::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:51:31,676::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:51:31,677::task::1185::TaskManager.Task::(prepare) Task=`4829c1bd-5074-4730-9aa9-b39efc9c92d2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:51:31,678::task::595::TaskManager.Task::(_updateState) Task=`4829c1bd-5074-4730-9aa9-b39efc9c92d2`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:51:31,679::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:51:31,680::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:51:31,681::task::990::TaskManager.Task::(_decref) Task=`4829c1bd-5074-4730-9aa9-b39efc9c92d2`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:51:31,689::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [1c80df6] >Thread-338::DEBUG::2014-05-13 12:51:31,690::task::595::TaskManager.Task::(_updateState) Task=`9bc77be1-65cb-448b-9713-3f739a1fb639`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:51:31,692::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:51:31,694::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1a9198a4-566f-49a0-9c01-3628905a03ba`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:51:31,695::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:51:31,696::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:51:31,697::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1a9198a4-566f-49a0-9c01-3628905a03ba`::Granted request >Thread-338::DEBUG::2014-05-13 12:51:31,699::task::827::TaskManager.Task::(resourceAcquired) Task=`9bc77be1-65cb-448b-9713-3f739a1fb639`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:51:31,700::task::990::TaskManager.Task::(_decref) Task=`9bc77be1-65cb-448b-9713-3f739a1fb639`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:51:31,705::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9598664704', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:51:31,706::task::1185::TaskManager.Task::(prepare) Task=`9bc77be1-65cb-448b-9713-3f739a1fb639`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9598664704', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:51:31,707::task::595::TaskManager.Task::(_updateState) Task=`9bc77be1-65cb-448b-9713-3f739a1fb639`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:51:31,708::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:51:31,709::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:51:31,710::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:51:31,711::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:51:31,712::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:51:31,713::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:51:31,715::task::990::TaskManager.Task::(_decref) Task=`9bc77be1-65cb-448b-9713-3f739a1fb639`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:51:31,753::task::595::TaskManager.Task::(_updateState) Task=`02d97c40-4829-4f83-91d7-006f8bf0faad`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:51:31,754::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:51:31,756::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000490838', 'lastCheck': '0.5', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000657313', 'lastCheck': '3.7', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000416569', 'lastCheck': '8.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:51:31,757::task::1185::TaskManager.Task::(prepare) Task=`02d97c40-4829-4f83-91d7-006f8bf0faad`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000490838', 'lastCheck': '0.5', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000657313', 'lastCheck': '3.7', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000416569', 'lastCheck': '8.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:51:31,758::task::595::TaskManager.Task::(_updateState) Task=`02d97c40-4829-4f83-91d7-006f8bf0faad`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:51:31,759::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:51:31,760::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:51:31,761::task::990::TaskManager.Task::(_decref) Task=`02d97c40-4829-4f83-91d7-006f8bf0faad`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:51:31,801::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0790706 s, 13.0 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:51:33,246::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:51:33,254::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000354978 s, 980 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:51:33,814::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:33,892::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353334 s, 29.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:51:34,541::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:51:35,905::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:35,995::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0477701 s, 21.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:51:37,562::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:51:38,009::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:38,083::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307917 s, 33.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:51:38,101::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:51:38,110::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000772641 s, 5.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:51:40,096::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:40,171::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306679 s, 33.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:51:40,585::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-357::DEBUG::2014-05-13 12:51:41,285::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:51:41,295::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000521002 s, 750 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:51:41,761::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17998964] >Thread-66::DEBUG::2014-05-13 12:51:41,762::task::595::TaskManager.Task::(_updateState) Task=`75a3b34e-204d-410a-a1a8-cb4a8bf4e6c5`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:51:41,762::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:51:41,762::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:51:41,763::task::1185::TaskManager.Task::(prepare) Task=`75a3b34e-204d-410a-a1a8-cb4a8bf4e6c5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:51:41,763::task::595::TaskManager.Task::(_updateState) Task=`75a3b34e-204d-410a-a1a8-cb4a8bf4e6c5`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:51:41,763::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:51:41,763::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:51:41,763::task::990::TaskManager.Task::(_decref) Task=`75a3b34e-204d-410a-a1a8-cb4a8bf4e6c5`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:51:41,770::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [17998964] >Thread-338::DEBUG::2014-05-13 12:51:41,770::task::595::TaskManager.Task::(_updateState) Task=`938b4619-aa2b-488e-8f30-f2018cbfaab9`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:51:41,770::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:51:41,771::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4d76186f-ff00-4236-ad29-e42019cc7c36`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:51:41,771::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:51:41,772::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:51:41,772::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4d76186f-ff00-4236-ad29-e42019cc7c36`::Granted request >Thread-338::DEBUG::2014-05-13 12:51:41,772::task::827::TaskManager.Task::(resourceAcquired) Task=`938b4619-aa2b-488e-8f30-f2018cbfaab9`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:51:41,772::task::990::TaskManager.Task::(_decref) Task=`938b4619-aa2b-488e-8f30-f2018cbfaab9`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:51:41,773::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9598664704', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:51:41,774::task::1185::TaskManager.Task::(prepare) Task=`938b4619-aa2b-488e-8f30-f2018cbfaab9`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9598664704', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:51:41,774::task::595::TaskManager.Task::(_updateState) Task=`938b4619-aa2b-488e-8f30-f2018cbfaab9`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:51:41,774::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:51:41,774::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:51:41,775::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:51:41,775::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:51:41,775::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:51:41,775::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:51:41,775::task::990::TaskManager.Task::(_decref) Task=`938b4619-aa2b-488e-8f30-f2018cbfaab9`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:51:42,184::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:42,282::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0372785 s, 27.5 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:51:43,268::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:51:43,281::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000340427 s, 1.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:51:43,603::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:51:44,295::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:44,385::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360019 s, 28.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:51:46,399::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:46,480::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0332738 s, 30.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:51:46,625::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:51:46,910::task::595::TaskManager.Task::(_updateState) Task=`ebab5fb1-9f0d-4298-ba1c-848b6730902a`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:51:46,910::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:51:46,910::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000521002', 'lastCheck': '5.6', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000772641', 'lastCheck': '8.8', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000340427', 'lastCheck': '3.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:51:46,910::task::1185::TaskManager.Task::(prepare) Task=`ebab5fb1-9f0d-4298-ba1c-848b6730902a`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000521002', 'lastCheck': '5.6', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000772641', 'lastCheck': '8.8', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000340427', 'lastCheck': '3.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:51:46,911::task::595::TaskManager.Task::(_updateState) Task=`ebab5fb1-9f0d-4298-ba1c-848b6730902a`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:51:46,911::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:51:46,911::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:51:46,911::task::990::TaskManager.Task::(_decref) Task=`ebab5fb1-9f0d-4298-ba1c-848b6730902a`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 12:51:48,119::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:51:48,127::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000585058 s, 7.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:51:48,494::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:48,578::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0416987 s, 24.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:51:49,644::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:51:50,591::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:50,681::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0468979 s, 21.8 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:51:51,307::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:51:51,316::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000674247 s, 580 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:51:51,814::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:51:51,815::task::595::TaskManager.Task::(_updateState) Task=`361408a9-6743-46a0-9b4c-51afe5a28e28`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:51:51,815::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:51:51,815::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:51:51,815::task::1185::TaskManager.Task::(prepare) Task=`361408a9-6743-46a0-9b4c-51afe5a28e28`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:51:51,816::task::595::TaskManager.Task::(_updateState) Task=`361408a9-6743-46a0-9b4c-51afe5a28e28`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:51:51,816::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:51:51,816::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:51:51,816::task::990::TaskManager.Task::(_decref) Task=`361408a9-6743-46a0-9b4c-51afe5a28e28`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:51:51,821::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:51:51,821::task::595::TaskManager.Task::(_updateState) Task=`6cc9c33a-d9f5-408e-8991-e32a90471f21`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:51:51,821::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:51:51,822::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ff20494d-e696-43ed-889f-87df83faa9b6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:51:51,822::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:51:51,822::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:51:51,822::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ff20494d-e696-43ed-889f-87df83faa9b6`::Granted request >Thread-338::DEBUG::2014-05-13 12:51:51,823::task::827::TaskManager.Task::(resourceAcquired) Task=`6cc9c33a-d9f5-408e-8991-e32a90471f21`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:51:51,823::task::990::TaskManager.Task::(_decref) Task=`6cc9c33a-d9f5-408e-8991-e32a90471f21`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:51:51,824::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9598664704', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:51:51,824::task::1185::TaskManager.Task::(prepare) Task=`6cc9c33a-d9f5-408e-8991-e32a90471f21`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9598664704', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:51:51,824::task::595::TaskManager.Task::(_updateState) Task=`6cc9c33a-d9f5-408e-8991-e32a90471f21`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:51:51,825::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:51:51,825::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:51:51,825::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:51:51,825::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:51:51,826::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:51:51,826::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:51:51,826::task::990::TaskManager.Task::(_decref) Task=`6cc9c33a-d9f5-408e-8991-e32a90471f21`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:51:52,662::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:51:52,694::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:52,767::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306073 s, 33.5 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:51:53,292::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:51:53,301::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000411586 s, 846 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:51:54,780::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:54,859::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0354165 s, 28.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:51:55,682::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:51:56,872::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:56,948::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0333179 s, 30.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:51:58,136::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:51:58,144::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000640829 s, 6.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:51:58,701::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:51:58,961::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:51:59,050::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0424499 s, 24.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:52:01,063::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:52:01,138::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0331951 s, 30.8 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:52:01,329::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:52:01,339::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000417307 s, 937 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:52:01,719::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:52:01,861::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:52:01,862::task::595::TaskManager.Task::(_updateState) Task=`884894f1-f6d2-4c82-8d63-b7d2eaceae73`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:52:01,862::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:52:01,863::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:52:01,863::task::1185::TaskManager.Task::(prepare) Task=`884894f1-f6d2-4c82-8d63-b7d2eaceae73`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:52:01,863::task::595::TaskManager.Task::(_updateState) Task=`884894f1-f6d2-4c82-8d63-b7d2eaceae73`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:52:01,863::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:52:01,863::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:52:01,863::task::990::TaskManager.Task::(_decref) Task=`884894f1-f6d2-4c82-8d63-b7d2eaceae73`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:52:01,868::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:52:01,868::task::595::TaskManager.Task::(_updateState) Task=`8fbe9fa8-3c5a-4fb8-8355-2db36b931f6f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:52:01,869::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:52:01,869::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fe0c19e5-8c47-4079-aea7-e35a41f5decc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:52:01,869::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:52:01,870::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:52:01,870::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fe0c19e5-8c47-4079-aea7-e35a41f5decc`::Granted request >Thread-338::DEBUG::2014-05-13 12:52:01,870::task::827::TaskManager.Task::(resourceAcquired) Task=`8fbe9fa8-3c5a-4fb8-8355-2db36b931f6f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:52:01,870::task::990::TaskManager.Task::(_decref) Task=`8fbe9fa8-3c5a-4fb8-8355-2db36b931f6f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:52:01,871::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9598664704', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:52:01,871::task::1185::TaskManager.Task::(prepare) Task=`8fbe9fa8-3c5a-4fb8-8355-2db36b931f6f`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9598664704', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:52:01,872::task::595::TaskManager.Task::(_updateState) Task=`8fbe9fa8-3c5a-4fb8-8355-2db36b931f6f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:52:01,872::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:52:01,872::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:52:01,872::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:52:01,873::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:52:01,873::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:52:01,873::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:52:01,873::task::990::TaskManager.Task::(_decref) Task=`8fbe9fa8-3c5a-4fb8-8355-2db36b931f6f`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:52:02,054::task::595::TaskManager.Task::(_updateState) Task=`b2d0f808-a373-4507-adfa-b4d44f662995`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:52:02,054::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:52:02,054::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000417307', 'lastCheck': '0.7', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000640829', 'lastCheck': '3.9', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000411586', 'lastCheck': '8.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:52:02,055::task::1185::TaskManager.Task::(prepare) Task=`b2d0f808-a373-4507-adfa-b4d44f662995`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000417307', 'lastCheck': '0.7', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000640829', 'lastCheck': '3.9', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000411586', 'lastCheck': '8.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:52:02,055::task::595::TaskManager.Task::(_updateState) Task=`b2d0f808-a373-4507-adfa-b4d44f662995`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:52:02,055::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:52:02,055::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:52:02,055::task::990::TaskManager.Task::(_decref) Task=`b2d0f808-a373-4507-adfa-b4d44f662995`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:52:03,151::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:52:03,231::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0370365 s, 27.6 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:52:03,313::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:52:03,322::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000366143 s, 950 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:52:04,738::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:52:05,244::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:52:05,337::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0512717 s, 20.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:52:07,350::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:52:07,426::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0332692 s, 30.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:52:07,756::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:52:08,153::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:52:08,165::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000719005 s, 5.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:52:09,439::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:52:09,530::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037616 s, 27.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:52:10,775::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-357::DEBUG::2014-05-13 12:52:11,351::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:52:11,360::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.00048857 s, 800 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:52:11,543::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:52:11,622::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0372154 s, 27.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:52:11,908::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:52:11,909::task::595::TaskManager.Task::(_updateState) Task=`972548e4-e2f0-4d6f-a2b3-c389a405b3c5`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:52:11,909::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:52:11,910::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:52:11,910::task::1185::TaskManager.Task::(prepare) Task=`972548e4-e2f0-4d6f-a2b3-c389a405b3c5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:52:11,910::task::595::TaskManager.Task::(_updateState) Task=`972548e4-e2f0-4d6f-a2b3-c389a405b3c5`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:52:11,910::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:52:11,910::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:52:11,911::task::990::TaskManager.Task::(_decref) Task=`972548e4-e2f0-4d6f-a2b3-c389a405b3c5`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:52:11,915::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:52:11,915::task::595::TaskManager.Task::(_updateState) Task=`88840b25-161b-4c52-95bc-92148cb12fa8`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:52:11,916::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:52:11,916::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a8bcfb00-b609-4d3d-b060-f334bf9bc863`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:52:11,916::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:52:11,917::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:52:11,917::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a8bcfb00-b609-4d3d-b060-f334bf9bc863`::Granted request >Thread-338::DEBUG::2014-05-13 12:52:11,917::task::827::TaskManager.Task::(resourceAcquired) Task=`88840b25-161b-4c52-95bc-92148cb12fa8`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:52:11,917::task::990::TaskManager.Task::(_decref) Task=`88840b25-161b-4c52-95bc-92148cb12fa8`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:52:11,918::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9598664704', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:52:11,919::task::1185::TaskManager.Task::(prepare) Task=`88840b25-161b-4c52-95bc-92148cb12fa8`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9598664704', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:52:11,919::task::595::TaskManager.Task::(_updateState) Task=`88840b25-161b-4c52-95bc-92148cb12fa8`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:52:11,919::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:52:11,919::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:52:11,920::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:52:11,920::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:52:11,920::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:52:11,920::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:52:11,920::task::990::TaskManager.Task::(_decref) Task=`88840b25-161b-4c52-95bc-92148cb12fa8`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:52:13,332::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:52:13,342::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000408428 s, 852 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:52:13,635::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:52:13,715::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0376595 s, 27.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:52:13,796::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:52:15,728::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:52:15,808::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379594 s, 27.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:52:16,815::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:52:17,203::task::595::TaskManager.Task::(_updateState) Task=`d473e4d1-1e90-4990-8b8d-4bc9202789db`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:52:17,203::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:52:17,204::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00048857', 'lastCheck': '5.8', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000719005', 'lastCheck': '9.0', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000408428', 'lastCheck': '3.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:52:17,204::task::1185::TaskManager.Task::(prepare) Task=`d473e4d1-1e90-4990-8b8d-4bc9202789db`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00048857', 'lastCheck': '5.8', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000719005', 'lastCheck': '9.0', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000408428', 'lastCheck': '3.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:52:17,204::task::595::TaskManager.Task::(_updateState) Task=`d473e4d1-1e90-4990-8b8d-4bc9202789db`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:52:17,204::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:52:17,204::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:52:17,205::task::990::TaskManager.Task::(_decref) Task=`d473e4d1-1e90-4990-8b8d-4bc9202789db`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:52:17,821::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-322::DEBUG::2014-05-13 12:52:18,175::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:52:18,192::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00080867 s, 5.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:52:18,275::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.412464 s, 2.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:52:19,841::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:52:20,288::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:52:20,379::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0350901 s, 29.2 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:52:21,371::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:52:21,381::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000644279 s, 607 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:52:21,958::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:52:21,958::task::595::TaskManager.Task::(_updateState) Task=`a368a172-8142-45d6-a024-4fff9fdcd3b9`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:52:21,959::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:52:21,959::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:52:21,959::task::1185::TaskManager.Task::(prepare) Task=`a368a172-8142-45d6-a024-4fff9fdcd3b9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:52:21,959::task::595::TaskManager.Task::(_updateState) Task=`a368a172-8142-45d6-a024-4fff9fdcd3b9`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:52:21,959::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:52:21,960::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:52:21,960::task::990::TaskManager.Task::(_decref) Task=`a368a172-8142-45d6-a024-4fff9fdcd3b9`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:52:21,965::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:52:21,965::task::595::TaskManager.Task::(_updateState) Task=`807efd79-be8d-4f53-bcc9-89c89d223b98`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:52:21,966::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:52:21,966::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`79497ca7-dc2c-4faf-b3b9-43d3f9693818`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:52:21,966::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:52:21,967::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:52:21,967::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`79497ca7-dc2c-4faf-b3b9-43d3f9693818`::Granted request >Thread-338::DEBUG::2014-05-13 12:52:21,967::task::827::TaskManager.Task::(resourceAcquired) Task=`807efd79-be8d-4f53-bcc9-89c89d223b98`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:52:21,967::task::990::TaskManager.Task::(_decref) Task=`807efd79-be8d-4f53-bcc9-89c89d223b98`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:52:21,968::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:52:21,969::task::1185::TaskManager.Task::(prepare) Task=`807efd79-be8d-4f53-bcc9-89c89d223b98`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:52:21,969::task::595::TaskManager.Task::(_updateState) Task=`807efd79-be8d-4f53-bcc9-89c89d223b98`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:52:21,969::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:52:21,969::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:52:21,969::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:52:21,970::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:52:21,970::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:52:21,970::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:52:21,970::task::990::TaskManager.Task::(_decref) Task=`807efd79-be8d-4f53-bcc9-89c89d223b98`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:52:22,392::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:52:22,469::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.034049 s, 30.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:52:22,859::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-344::DEBUG::2014-05-13 12:52:23,353::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:52:23,362::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000398296 s, 874 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:52:24,482::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:52:24,569::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0386954 s, 26.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:52:25,878::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:52:26,585::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:52:26,662::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0345324 s, 29.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:52:28,222::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:52:28,230::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000583012 s, 7.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:52:28,675::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:52:28,753::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0350391 s, 29.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:52:28,896::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:52:30,766::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:52:30,860::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0493314 s, 20.8 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:52:31,391::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:52:31,402::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.00131312 s, 298 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:52:31,914::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:52:32,011::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:52:32,011::task::595::TaskManager.Task::(_updateState) Task=`82a6bdba-63fa-4458-bf81-1bf568e9ae72`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:52:32,012::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:52:32,012::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:52:32,012::task::1185::TaskManager.Task::(prepare) Task=`82a6bdba-63fa-4458-bf81-1bf568e9ae72`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:52:32,012::task::595::TaskManager.Task::(_updateState) Task=`82a6bdba-63fa-4458-bf81-1bf568e9ae72`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:52:32,013::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:52:32,013::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:52:32,013::task::990::TaskManager.Task::(_decref) Task=`82a6bdba-63fa-4458-bf81-1bf568e9ae72`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:52:32,017::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:52:32,018::task::595::TaskManager.Task::(_updateState) Task=`b8ec3f35-2460-4234-b6c2-c57f3f7d542a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:52:32,018::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:52:32,018::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`00bc489b-fe65-4b5c-bb8b-69cf4d9e76c1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:52:32,020::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:52:32,020::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:52:32,020::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`00bc489b-fe65-4b5c-bb8b-69cf4d9e76c1`::Granted request >Thread-338::DEBUG::2014-05-13 12:52:32,020::task::827::TaskManager.Task::(resourceAcquired) Task=`b8ec3f35-2460-4234-b6c2-c57f3f7d542a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:52:32,021::task::990::TaskManager.Task::(_decref) Task=`b8ec3f35-2460-4234-b6c2-c57f3f7d542a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:52:32,021::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:52:32,022::task::1185::TaskManager.Task::(prepare) Task=`b8ec3f35-2460-4234-b6c2-c57f3f7d542a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:52:32,022::task::595::TaskManager.Task::(_updateState) Task=`b8ec3f35-2460-4234-b6c2-c57f3f7d542a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:52:32,022::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:52:32,022::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:52:32,023::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:52:32,023::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:52:32,023::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:52:32,023::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:52:32,023::task::990::TaskManager.Task::(_decref) Task=`b8ec3f35-2460-4234-b6c2-c57f3f7d542a`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:52:32,336::task::595::TaskManager.Task::(_updateState) Task=`8b453d81-a8b9-4df8-a744-bceed5f130ce`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:52:32,336::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:52:32,336::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00131312', 'lastCheck': '0.9', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000583012', 'lastCheck': '4.1', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000398296', 'lastCheck': '9.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:52:32,336::task::1185::TaskManager.Task::(prepare) Task=`8b453d81-a8b9-4df8-a744-bceed5f130ce`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00131312', 'lastCheck': '0.9', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000583012', 'lastCheck': '4.1', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000398296', 'lastCheck': '9.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:52:32,337::task::595::TaskManager.Task::(_updateState) Task=`8b453d81-a8b9-4df8-a744-bceed5f130ce`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:52:32,337::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:52:32,337::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:52:32,337::task::990::TaskManager.Task::(_decref) Task=`8b453d81-a8b9-4df8-a744-bceed5f130ce`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:52:32,874::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:52:32,956::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363994 s, 28.1 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:52:33,373::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:52:33,382::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000399493 s, 871 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:52:34,938::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:52:34,969::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:52:35,051::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0377466 s, 27.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:52:37,064::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:52:37,137::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303896 s, 33.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:52:37,962::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:52:38,239::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:52:38,247::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00062605 s, 6.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:52:39,150::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:52:39,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357969 s, 28.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:52:40,984::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:52:41,242::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:52:41,317::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310847 s, 32.9 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:52:41,414::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:52:41,424::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000459232 s, 851 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:52:42,062::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:52:42,063::task::595::TaskManager.Task::(_updateState) Task=`5f8885de-a43c-4eda-b252-591d2e221ce6`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:52:42,063::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:52:42,063::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:52:42,064::task::1185::TaskManager.Task::(prepare) Task=`5f8885de-a43c-4eda-b252-591d2e221ce6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:52:42,095::task::595::TaskManager.Task::(_updateState) Task=`5f8885de-a43c-4eda-b252-591d2e221ce6`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:52:42,095::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:52:42,095::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:52:42,095::task::990::TaskManager.Task::(_decref) Task=`5f8885de-a43c-4eda-b252-591d2e221ce6`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:52:42,100::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:52:42,101::task::595::TaskManager.Task::(_updateState) Task=`5643b787-42e8-4139-8aed-2a2ffd83e093`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:52:42,101::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:52:42,101::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fa7cb09b-d645-4588-8eac-0a2ac4fae243`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:52:42,102::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:52:42,102::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:52:42,102::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fa7cb09b-d645-4588-8eac-0a2ac4fae243`::Granted request >Thread-338::DEBUG::2014-05-13 12:52:42,103::task::827::TaskManager.Task::(resourceAcquired) Task=`5643b787-42e8-4139-8aed-2a2ffd83e093`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:52:42,103::task::990::TaskManager.Task::(_decref) Task=`5643b787-42e8-4139-8aed-2a2ffd83e093`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:52:42,104::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:52:42,104::task::1185::TaskManager.Task::(prepare) Task=`5643b787-42e8-4139-8aed-2a2ffd83e093`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:52:42,104::task::595::TaskManager.Task::(_updateState) Task=`5643b787-42e8-4139-8aed-2a2ffd83e093`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:52:42,104::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:52:42,104::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:52:42,105::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:52:42,105::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:52:42,105::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:52:42,105::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:52:42,106::task::990::TaskManager.Task::(_decref) Task=`5643b787-42e8-4139-8aed-2a2ffd83e093`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:52:43,330::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-344::DEBUG::2014-05-13 12:52:43,435::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:52:43,455::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000363775 s, 957 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:52:43,476::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0304203 s, 33.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:52:44,010::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:52:45,489::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:52:45,900::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.366624 s, 2.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:52:47,028::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:52:47,468::task::595::TaskManager.Task::(_updateState) Task=`76051365-07c9-4738-90bd-397deab16aa3`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:52:47,469::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:52:47,469::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000459232', 'lastCheck': '6.0', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00062605', 'lastCheck': '9.2', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000363775', 'lastCheck': '4.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:52:47,469::task::1185::TaskManager.Task::(prepare) Task=`76051365-07c9-4738-90bd-397deab16aa3`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000459232', 'lastCheck': '6.0', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00062605', 'lastCheck': '9.2', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000363775', 'lastCheck': '4.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:52:47,469::task::595::TaskManager.Task::(_updateState) Task=`76051365-07c9-4738-90bd-397deab16aa3`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:52:47,470::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:52:47,470::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:52:47,470::task::990::TaskManager.Task::(_decref) Task=`76051365-07c9-4738-90bd-397deab16aa3`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:52:47,913::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:52:47,988::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313727 s, 32.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:52:48,256::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:52:48,265::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00132882 s, 3.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:52:50,002::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:52:50,056::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:52:50,084::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365103 s, 28.0 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:52:51,436::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:52:51,447::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.00209364 s, 187 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:52:52,097::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 12:52:52,146::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4765f2fb] >Thread-66::DEBUG::2014-05-13 12:52:52,148::task::595::TaskManager.Task::(_updateState) Task=`d6b1cecc-89a8-4e33-8e48-448d852a89b5`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:52:52,149::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:52:52,151::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:52:52,152::task::1185::TaskManager.Task::(prepare) Task=`d6b1cecc-89a8-4e33-8e48-448d852a89b5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:52:52,153::task::595::TaskManager.Task::(_updateState) Task=`d6b1cecc-89a8-4e33-8e48-448d852a89b5`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:52:52,153::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:52:52,154::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:52:52,155::task::990::TaskManager.Task::(_decref) Task=`d6b1cecc-89a8-4e33-8e48-448d852a89b5`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:52:52,162::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4765f2fb] >Thread-338::DEBUG::2014-05-13 12:52:52,164::task::595::TaskManager.Task::(_updateState) Task=`1816476a-3799-4924-926f-ce9b0e579199`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:52:52,165::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:52:52,167::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9656191a-e622-4e0c-9244-236dd706581d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:52:52,168::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:52:52,170::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:52:52,171::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9656191a-e622-4e0c-9244-236dd706581d`::Granted request >Thread-338::DEBUG::2014-05-13 12:52:52,173::task::827::TaskManager.Task::(resourceAcquired) Task=`1816476a-3799-4924-926f-ce9b0e579199`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:52:52,174::task::990::TaskManager.Task::(_decref) Task=`1816476a-3799-4924-926f-ce9b0e579199`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:52:52,178::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:52:52,179::task::1185::TaskManager.Task::(prepare) Task=`1816476a-3799-4924-926f-ce9b0e579199`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:52:52,180::task::595::TaskManager.Task::(_updateState) Task=`1816476a-3799-4924-926f-ce9b0e579199`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:52:52,181::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:52:52,182::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:52:52,183::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:52:52,185::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:52:52,186::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:52:52,186::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:52:52,188::task::990::TaskManager.Task::(_decref) Task=`1816476a-3799-4924-926f-ce9b0e579199`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:52:52,213::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312642 s, 32.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:52:53,073::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-344::DEBUG::2014-05-13 12:52:53,471::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:52:53,480::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.00043217 s, 805 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:52:54,226::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:52:54,305::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0357602 s, 28.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:52:56,094::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:52:56,318::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:52:56,394::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315552 s, 32.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:52:58,273::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:52:58,282::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000711252 s, 5.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:52:58,408::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:52:58,859::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.410044 s, 2.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:52:59,113::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:53:00,872::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:53:00,955::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0377931 s, 27.1 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:53:01,459::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:53:01,469::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000476877 s, 820 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:53:02,132::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:53:02,243::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [50b954f9] >Thread-66::DEBUG::2014-05-13 12:53:02,243::task::595::TaskManager.Task::(_updateState) Task=`71d31919-4b26-4046-83a5-afa61eff71d3`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:53:02,243::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:53:02,244::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:53:02,244::task::1185::TaskManager.Task::(prepare) Task=`71d31919-4b26-4046-83a5-afa61eff71d3`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:53:02,244::task::595::TaskManager.Task::(_updateState) Task=`71d31919-4b26-4046-83a5-afa61eff71d3`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:53:02,244::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:53:02,245::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:53:02,245::task::990::TaskManager.Task::(_decref) Task=`71d31919-4b26-4046-83a5-afa61eff71d3`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:53:02,513::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [50b954f9] >Thread-338::DEBUG::2014-05-13 12:53:02,514::task::595::TaskManager.Task::(_updateState) Task=`0b4e4741-95f7-41ba-83a2-f961cc653bf2`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:53:02,514::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:53:02,514::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ba5a5297-9c80-4d33-acc6-8902c160581c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:53:02,514::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:53:02,515::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:53:02,515::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ba5a5297-9c80-4d33-acc6-8902c160581c`::Granted request >Thread-338::DEBUG::2014-05-13 12:53:02,515::task::827::TaskManager.Task::(resourceAcquired) Task=`0b4e4741-95f7-41ba-83a2-f961cc653bf2`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:53:02,515::task::990::TaskManager.Task::(_decref) Task=`0b4e4741-95f7-41ba-83a2-f961cc653bf2`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:53:02,516::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:53:02,517::task::1185::TaskManager.Task::(prepare) Task=`0b4e4741-95f7-41ba-83a2-f961cc653bf2`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:53:02,517::task::595::TaskManager.Task::(_updateState) Task=`0b4e4741-95f7-41ba-83a2-f961cc653bf2`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:53:02,517::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:53:02,517::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:53:02,517::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:53:02,518::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:53:02,518::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:53:02,518::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:53:02,518::task::990::TaskManager.Task::(_decref) Task=`0b4e4741-95f7-41ba-83a2-f961cc653bf2`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:53:02,601::task::595::TaskManager.Task::(_updateState) Task=`b7f1ac9e-2e8b-4d96-bc85-0feae35dddfc`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:53:02,601::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:53:02,601::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000476877', 'lastCheck': '1.1', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000711252', 'lastCheck': '4.3', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00043217', 'lastCheck': '9.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:53:02,602::task::1185::TaskManager.Task::(prepare) Task=`b7f1ac9e-2e8b-4d96-bc85-0feae35dddfc`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000476877', 'lastCheck': '1.1', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000711252', 'lastCheck': '4.3', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00043217', 'lastCheck': '9.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:53:02,602::task::595::TaskManager.Task::(_updateState) Task=`b7f1ac9e-2e8b-4d96-bc85-0feae35dddfc`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:53:02,602::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:53:02,602::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:53:02,602::task::990::TaskManager.Task::(_decref) Task=`b7f1ac9e-2e8b-4d96-bc85-0feae35dddfc`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:53:02,968::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:53:03,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0379716 s, 27.0 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:53:03,492::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:53:03,501::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000500031 s, 696 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:53:05,061::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:53:05,142::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381961 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:53:05,152::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:53:07,159::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:53:07,234::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.032762 s, 31.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:53:08,171::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:53:08,290::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:53:08,299::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000628448 s, 6.5 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:53:09,247::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:53:09,326::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0352068 s, 29.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:53:11,191::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:53:11,339::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-357::DEBUG::2014-05-13 12:53:11,480::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:53:11,489::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.00053207 s, 735 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:53:11,709::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.326475 s, 3.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:53:12,568::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:53:12,568::task::595::TaskManager.Task::(_updateState) Task=`ab0fad12-e894-4b35-9ea8-edd546e498d3`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:53:12,568::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:53:12,569::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:53:12,569::task::1185::TaskManager.Task::(prepare) Task=`ab0fad12-e894-4b35-9ea8-edd546e498d3`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:53:12,569::task::595::TaskManager.Task::(_updateState) Task=`ab0fad12-e894-4b35-9ea8-edd546e498d3`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:53:12,569::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:53:12,570::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:53:12,570::task::990::TaskManager.Task::(_decref) Task=`ab0fad12-e894-4b35-9ea8-edd546e498d3`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:53:12,575::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:53:12,575::task::595::TaskManager.Task::(_updateState) Task=`8d159889-b655-43c3-b1a3-e3de9e3b2771`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:53:12,575::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:53:12,576::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`50913547-01f0-4f2a-a2f5-61620501b250`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:53:12,576::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:53:12,576::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:53:12,577::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`50913547-01f0-4f2a-a2f5-61620501b250`::Granted request >Thread-338::DEBUG::2014-05-13 12:53:12,577::task::827::TaskManager.Task::(resourceAcquired) Task=`8d159889-b655-43c3-b1a3-e3de9e3b2771`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:53:12,577::task::990::TaskManager.Task::(_decref) Task=`8d159889-b655-43c3-b1a3-e3de9e3b2771`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:53:12,578::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:53:12,579::task::1185::TaskManager.Task::(prepare) Task=`8d159889-b655-43c3-b1a3-e3de9e3b2771`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:53:12,579::task::595::TaskManager.Task::(_updateState) Task=`8d159889-b655-43c3-b1a3-e3de9e3b2771`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:53:12,579::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:53:12,580::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:53:12,580::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:53:12,581::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:53:12,581::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:53:12,581::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:53:12,582::task::990::TaskManager.Task::(_decref) Task=`8d159889-b655-43c3-b1a3-e3de9e3b2771`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:53:13,504::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 572b0496-5eab-41cd-a976-4514f111898e >Thread-344::DEBUG::2014-05-13 12:53:13,509::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e >Thread-344::DEBUG::2014-05-13 12:53:13,510::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-344::DEBUG::2014-05-13 12:53:13,517::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=NFS_2', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', 'REMOTE_PATH=10.35.160.108:/RHEV/ogofen/3', 'ROLE=Regular', 'SDUUID=572b0496-5eab-41cd-a976-4514f111898e', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=06893a9d3c8506053131030a281fb9b9deb4efa9'] >Thread-344::DEBUG::2014-05-13 12:53:13,519::fileSD::609::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-344::INFO::2014-05-13 12:53:13,519::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 572b0496-5eab-41cd-a976-4514f111898e_imageNS already registered >Thread-344::INFO::2014-05-13 12:53:13,519::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 572b0496-5eab-41cd-a976-4514f111898e_volumeNS already registered >Thread-344::DEBUG::2014-05-13 12:53:13,527::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:53:13,535::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000523145 s, 665 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:53:13,721::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:53:13,805::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0407395 s, 25.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:53:14,209::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:53:15,818::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:53:15,898::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364789 s, 28.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:53:17,227::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:53:17,911::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:53:17,988::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319539 s, 32.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:53:18,116::task::595::TaskManager.Task::(_updateState) Task=`f1123a48-ef66-4011-84ab-4f1d1998278d`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:53:18,116::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:53:18,116::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00053207', 'lastCheck': '6.6', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000628448', 'lastCheck': '9.8', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000523145', 'lastCheck': '4.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:53:18,117::task::1185::TaskManager.Task::(prepare) Task=`f1123a48-ef66-4011-84ab-4f1d1998278d`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00053207', 'lastCheck': '6.6', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000628448', 'lastCheck': '9.8', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000523145', 'lastCheck': '4.6', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:53:18,117::task::595::TaskManager.Task::(_updateState) Task=`f1123a48-ef66-4011-84ab-4f1d1998278d`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:53:18,117::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:53:18,117::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:53:18,117::task::990::TaskManager.Task::(_decref) Task=`f1123a48-ef66-4011-84ab-4f1d1998278d`::ref 0 aborting False >Thread-322::DEBUG::2014-05-13 12:53:18,307::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:53:18,316::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000556349 s, 7.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:53:20,001::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:53:20,080::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355735 s, 28.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:53:20,245::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-357::DEBUG::2014-05-13 12:53:21,501::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:53:21,510::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000448643 s, 872 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:53:22,093::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:53:22,168::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313732 s, 32.6 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:53:22,618::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:53:22,618::task::595::TaskManager.Task::(_updateState) Task=`3d39fc85-1508-45f6-813e-98d029ae6313`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:53:22,619::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:53:22,619::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:53:22,619::task::1185::TaskManager.Task::(prepare) Task=`3d39fc85-1508-45f6-813e-98d029ae6313`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:53:22,619::task::595::TaskManager.Task::(_updateState) Task=`3d39fc85-1508-45f6-813e-98d029ae6313`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:53:22,620::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:53:22,620::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:53:22,620::task::990::TaskManager.Task::(_decref) Task=`3d39fc85-1508-45f6-813e-98d029ae6313`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:53:22,625::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:53:22,625::task::595::TaskManager.Task::(_updateState) Task=`856f8771-9ba2-4789-8177-8057eb1c7adb`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:53:22,625::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:53:22,626::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8b0537e1-c2a3-40fb-bd4a-99c977bc8eb6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:53:22,626::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:53:22,626::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:53:22,626::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`8b0537e1-c2a3-40fb-bd4a-99c977bc8eb6`::Granted request >Thread-338::DEBUG::2014-05-13 12:53:22,627::task::827::TaskManager.Task::(resourceAcquired) Task=`856f8771-9ba2-4789-8177-8057eb1c7adb`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:53:22,627::task::990::TaskManager.Task::(_decref) Task=`856f8771-9ba2-4789-8177-8057eb1c7adb`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:53:22,628::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:53:22,628::task::1185::TaskManager.Task::(prepare) Task=`856f8771-9ba2-4789-8177-8057eb1c7adb`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:53:22,628::task::595::TaskManager.Task::(_updateState) Task=`856f8771-9ba2-4789-8177-8057eb1c7adb`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:53:22,628::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:53:22,629::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:53:22,629::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:53:22,629::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:53:22,629::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:53:22,629::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:53:22,630::task::990::TaskManager.Task::(_decref) Task=`856f8771-9ba2-4789-8177-8057eb1c7adb`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:53:23,264::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-344::DEBUG::2014-05-13 12:53:23,547::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:53:23,556::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000397348 s, 876 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:53:24,181::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:53:24,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0536514 s, 19.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:53:26,281::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:53:26,291::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:53:26,367::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0317416 s, 32.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:53:28,323::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:53:28,332::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00076797 s, 5.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:53:28,380::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:53:28,461::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365776 s, 28.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:53:29,299::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:53:30,474::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:53:30,575::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0386159 s, 26.5 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:53:31,521::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:53:31,530::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000471604 s, 829 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:53:32,319::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:53:32,588::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 12:53:32,667::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:53:32,669::task::595::TaskManager.Task::(_updateState) Task=`60b1100d-7e16-42a3-a1b0-239543ecbeca`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:53:32,670::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:53:32,672::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Dummy-337::DEBUG::2014-05-13 12:53:32,672::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0350399 s, 29.2 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:53:32,673::task::1185::TaskManager.Task::(prepare) Task=`60b1100d-7e16-42a3-a1b0-239543ecbeca`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:53:32,674::task::595::TaskManager.Task::(_updateState) Task=`60b1100d-7e16-42a3-a1b0-239543ecbeca`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:53:32,675::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:53:32,675::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:53:32,676::task::990::TaskManager.Task::(_decref) Task=`60b1100d-7e16-42a3-a1b0-239543ecbeca`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:53:32,685::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:53:32,686::task::595::TaskManager.Task::(_updateState) Task=`a668fd90-cd57-4bd8-8d55-53de2c8b9f61`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:53:32,687::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:53:32,689::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`52391ca2-56eb-4267-ac37-995af197123a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:53:32,690::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:53:32,691::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:53:32,692::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`52391ca2-56eb-4267-ac37-995af197123a`::Granted request >Thread-338::DEBUG::2014-05-13 12:53:32,694::task::827::TaskManager.Task::(resourceAcquired) Task=`a668fd90-cd57-4bd8-8d55-53de2c8b9f61`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:53:32,694::task::990::TaskManager.Task::(_decref) Task=`a668fd90-cd57-4bd8-8d55-53de2c8b9f61`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:53:32,698::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:53:32,699::task::1185::TaskManager.Task::(prepare) Task=`a668fd90-cd57-4bd8-8d55-53de2c8b9f61`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:53:32,700::task::595::TaskManager.Task::(_updateState) Task=`a668fd90-cd57-4bd8-8d55-53de2c8b9f61`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:53:32,701::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:53:32,701::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:53:32,702::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:53:32,703::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:53:32,704::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:53:32,705::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:53:32,706::task::990::TaskManager.Task::(_decref) Task=`a668fd90-cd57-4bd8-8d55-53de2c8b9f61`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:53:33,257::task::595::TaskManager.Task::(_updateState) Task=`3d1bf92b-3dfb-41c6-bae4-a9641e7b3f90`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:53:33,258::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:53:33,258::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000471604', 'lastCheck': '1.7', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00076797', 'lastCheck': '4.9', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000397348', 'lastCheck': '9.7', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:53:33,258::task::1185::TaskManager.Task::(prepare) Task=`3d1bf92b-3dfb-41c6-bae4-a9641e7b3f90`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000471604', 'lastCheck': '1.7', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00076797', 'lastCheck': '4.9', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000397348', 'lastCheck': '9.7', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:53:33,258::task::595::TaskManager.Task::(_updateState) Task=`3d1bf92b-3dfb-41c6-bae4-a9641e7b3f90`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:53:33,259::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:53:33,259::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:53:33,259::task::990::TaskManager.Task::(_decref) Task=`3d1bf92b-3dfb-41c6-bae4-a9641e7b3f90`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:53:33,567::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:53:33,575::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000406039 s, 857 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:53:34,716::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:53:34,791::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313296 s, 32.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:53:35,338::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:53:36,804::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:53:37,161::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.314164 s, 3.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:53:38,341::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:53:38,349::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000729927 s, 5.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:53:38,358::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:53:39,174::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:53:39,258::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0387916 s, 26.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:53:41,271::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:53:41,345::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306256 s, 33.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:53:41,377::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-357::DEBUG::2014-05-13 12:53:41,541::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:53:41,551::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000518545 s, 754 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:53:42,749::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:53:42,749::task::595::TaskManager.Task::(_updateState) Task=`f59d898e-39c0-441f-b6bf-b2df965d2d20`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:53:42,750::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:53:42,750::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:53:42,750::task::1185::TaskManager.Task::(prepare) Task=`f59d898e-39c0-441f-b6bf-b2df965d2d20`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:53:42,750::task::595::TaskManager.Task::(_updateState) Task=`f59d898e-39c0-441f-b6bf-b2df965d2d20`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:53:42,751::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:53:42,751::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:53:42,751::task::990::TaskManager.Task::(_decref) Task=`f59d898e-39c0-441f-b6bf-b2df965d2d20`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:53:42,755::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:53:42,756::task::595::TaskManager.Task::(_updateState) Task=`96e7d715-4df5-48ea-9a49-d2dec7d89be4`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:53:42,756::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:53:42,756::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`817ecf9a-7851-49b8-9bfa-9b7baa8fae85`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:53:42,757::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:53:42,757::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:53:42,757::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`817ecf9a-7851-49b8-9bfa-9b7baa8fae85`::Granted request >Thread-338::DEBUG::2014-05-13 12:53:42,758::task::827::TaskManager.Task::(resourceAcquired) Task=`96e7d715-4df5-48ea-9a49-d2dec7d89be4`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:53:42,758::task::990::TaskManager.Task::(_decref) Task=`96e7d715-4df5-48ea-9a49-d2dec7d89be4`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:53:42,759::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:53:42,759::task::1185::TaskManager.Task::(prepare) Task=`96e7d715-4df5-48ea-9a49-d2dec7d89be4`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:53:42,759::task::595::TaskManager.Task::(_updateState) Task=`96e7d715-4df5-48ea-9a49-d2dec7d89be4`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:53:42,759::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:53:42,760::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:53:42,760::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:53:42,760::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:53:42,760::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:53:42,760::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:53:42,761::task::990::TaskManager.Task::(_decref) Task=`96e7d715-4df5-48ea-9a49-d2dec7d89be4`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:53:43,358::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:53:43,434::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0336916 s, 30.4 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:53:43,586::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:53:43,596::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000436336 s, 798 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:53:44,397::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:53:45,447::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:53:45,529::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383621 s, 26.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:53:47,418::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:53:47,542::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:53:47,617::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320923 s, 31.9 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:53:48,358::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:53:48,366::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000619448 s, 6.6 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:53:48,397::task::595::TaskManager.Task::(_updateState) Task=`fa8f2ae0-9a0e-49f2-98b6-b79e11b7355a`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:53:48,397::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:53:48,398::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000518545', 'lastCheck': '6.8', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000619448', 'lastCheck': '0.0', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000436336', 'lastCheck': '4.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:53:48,398::task::1185::TaskManager.Task::(prepare) Task=`fa8f2ae0-9a0e-49f2-98b6-b79e11b7355a`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000518545', 'lastCheck': '6.8', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000619448', 'lastCheck': '0.0', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000436336', 'lastCheck': '4.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:53:48,398::task::595::TaskManager.Task::(_updateState) Task=`fa8f2ae0-9a0e-49f2-98b6-b79e11b7355a`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:53:48,398::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:53:48,399::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:53:48,399::task::990::TaskManager.Task::(_decref) Task=`fa8f2ae0-9a0e-49f2-98b6-b79e11b7355a`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:53:49,630::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:53:49,704::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311483 s, 32.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:53:50,436::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-357::DEBUG::2014-05-13 12:53:51,563::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:53:51,572::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000515824 s, 758 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:53:51,717::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:53:51,810::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0502147 s, 20.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:53:52,800::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:53:52,800::task::595::TaskManager.Task::(_updateState) Task=`21af3043-3438-452a-a6cc-aec80ca6f645`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:53:52,800::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:53:52,801::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:53:52,801::task::1185::TaskManager.Task::(prepare) Task=`21af3043-3438-452a-a6cc-aec80ca6f645`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:53:52,801::task::595::TaskManager.Task::(_updateState) Task=`21af3043-3438-452a-a6cc-aec80ca6f645`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:53:52,801::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:53:52,801::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:53:52,801::task::990::TaskManager.Task::(_decref) Task=`21af3043-3438-452a-a6cc-aec80ca6f645`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:53:52,806::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:53:52,807::task::595::TaskManager.Task::(_updateState) Task=`2b7216e5-5c34-49ca-8f34-576fe0d62c7a`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:53:52,807::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:53:52,807::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a255280b-c46b-4ecb-9c21-c9485dee9f44`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:53:52,807::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:53:52,808::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:53:52,808::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`a255280b-c46b-4ecb-9c21-c9485dee9f44`::Granted request >Thread-338::DEBUG::2014-05-13 12:53:52,808::task::827::TaskManager.Task::(resourceAcquired) Task=`2b7216e5-5c34-49ca-8f34-576fe0d62c7a`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:53:52,808::task::990::TaskManager.Task::(_decref) Task=`2b7216e5-5c34-49ca-8f34-576fe0d62c7a`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:53:52,809::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:53:52,809::task::1185::TaskManager.Task::(prepare) Task=`2b7216e5-5c34-49ca-8f34-576fe0d62c7a`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:53:52,810::task::595::TaskManager.Task::(_updateState) Task=`2b7216e5-5c34-49ca-8f34-576fe0d62c7a`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:53:52,810::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:53:52,810::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:53:52,810::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:53:52,810::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:53:52,810::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:53:52,811::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:53:52,811::task::990::TaskManager.Task::(_decref) Task=`2b7216e5-5c34-49ca-8f34-576fe0d62c7a`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:53:53,457::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-344::DEBUG::2014-05-13 12:53:53,606::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:53:53,616::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000810801 s, 429 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:53:53,823::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:53:53,906::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0377275 s, 27.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:53:55,919::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:53:55,992::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306795 s, 33.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:53:56,475::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:53:58,006::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:53:58,087::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0362581 s, 28.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:53:58,387::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:53:58,395::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000655673 s, 6.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:53:59,502::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:54:00,097::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:00,179::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373442 s, 27.4 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:54:01,584::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:54:01,593::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000570972 s, 685 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:54:02,193::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:02,272::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359209 s, 28.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:54:02,520::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:54:02,864::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [dc1f215] >Thread-66::DEBUG::2014-05-13 12:54:02,865::task::595::TaskManager.Task::(_updateState) Task=`a69a0645-0c3a-4cb7-b19e-8387f7b6ed57`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:54:02,865::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:54:02,866::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:54:02,866::task::1185::TaskManager.Task::(prepare) Task=`a69a0645-0c3a-4cb7-b19e-8387f7b6ed57`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:54:02,866::task::595::TaskManager.Task::(_updateState) Task=`a69a0645-0c3a-4cb7-b19e-8387f7b6ed57`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:54:02,866::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:54:02,866::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:54:02,867::task::990::TaskManager.Task::(_decref) Task=`a69a0645-0c3a-4cb7-b19e-8387f7b6ed57`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:54:02,873::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [dc1f215] >Thread-338::DEBUG::2014-05-13 12:54:02,874::task::595::TaskManager.Task::(_updateState) Task=`5e61d4a5-7ad1-4469-bb71-080c579665f9`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:54:02,874::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:54:02,874::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1ce22e93-ce6b-4245-899d-d56ed69c5846`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:54:02,874::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:54:02,875::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:54:02,875::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1ce22e93-ce6b-4245-899d-d56ed69c5846`::Granted request >Thread-338::DEBUG::2014-05-13 12:54:02,875::task::827::TaskManager.Task::(resourceAcquired) Task=`5e61d4a5-7ad1-4469-bb71-080c579665f9`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:54:02,875::task::990::TaskManager.Task::(_decref) Task=`5e61d4a5-7ad1-4469-bb71-080c579665f9`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:54:02,876::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:54:02,877::task::1185::TaskManager.Task::(prepare) Task=`5e61d4a5-7ad1-4469-bb71-080c579665f9`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:54:02,877::task::595::TaskManager.Task::(_updateState) Task=`5e61d4a5-7ad1-4469-bb71-080c579665f9`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:54:02,877::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:54:02,877::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:54:02,877::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:54:02,878::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:54:02,878::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:54:02,878::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:54:02,878::task::990::TaskManager.Task::(_decref) Task=`5e61d4a5-7ad1-4469-bb71-080c579665f9`::ref 0 aborting False >Thread-66::DEBUG::2014-05-13 12:54:03,533::task::595::TaskManager.Task::(_updateState) Task=`de289bd1-f04b-44f9-bf02-022788d08518`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:54:03,534::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:54:03,534::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000570972', 'lastCheck': '1.9', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000655673', 'lastCheck': '5.1', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000810801', 'lastCheck': '9.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:54:03,534::task::1185::TaskManager.Task::(prepare) Task=`de289bd1-f04b-44f9-bf02-022788d08518`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000570972', 'lastCheck': '1.9', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000655673', 'lastCheck': '5.1', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000810801', 'lastCheck': '9.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:54:03,534::task::595::TaskManager.Task::(_updateState) Task=`de289bd1-f04b-44f9-bf02-022788d08518`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:54:03,534::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:54:03,535::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:54:03,535::task::990::TaskManager.Task::(_decref) Task=`de289bd1-f04b-44f9-bf02-022788d08518`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:54:03,627::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:54:03,636::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.00047278 s, 736 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:54:04,285::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:04,360::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310837 s, 32.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:54:05,543::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:54:06,373::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:06,469::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0502674 s, 20.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:54:08,403::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:54:08,412::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000654258 s, 6.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:54:08,482::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:08,561::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.035422 s, 28.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:54:08,562::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:54:10,577::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:10,672::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0306954 s, 33.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:54:11,580::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-357::DEBUG::2014-05-13 12:54:11,606::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:54:11,615::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000397805 s, 983 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:54:12,685::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:12,764::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0351824 s, 29.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:54:12,922::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [7d725256] >Thread-66::DEBUG::2014-05-13 12:54:12,923::task::595::TaskManager.Task::(_updateState) Task=`ef5481f7-79dc-4293-92e7-30091194dfcf`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:54:12,923::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:54:12,924::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:54:12,924::task::1185::TaskManager.Task::(prepare) Task=`ef5481f7-79dc-4293-92e7-30091194dfcf`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:54:12,924::task::595::TaskManager.Task::(_updateState) Task=`ef5481f7-79dc-4293-92e7-30091194dfcf`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:54:12,924::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:54:12,924::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:54:12,925::task::990::TaskManager.Task::(_decref) Task=`ef5481f7-79dc-4293-92e7-30091194dfcf`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:54:12,929::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [7d725256] >Thread-338::DEBUG::2014-05-13 12:54:12,929::task::595::TaskManager.Task::(_updateState) Task=`b40d2bc3-5b8d-4840-874b-f45c234e5b73`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:54:12,930::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:54:12,930::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e37139bc-0b51-4bc5-be82-9fe148c28a8a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:54:12,930::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:54:12,931::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:54:12,931::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e37139bc-0b51-4bc5-be82-9fe148c28a8a`::Granted request >Thread-338::DEBUG::2014-05-13 12:54:12,931::task::827::TaskManager.Task::(resourceAcquired) Task=`b40d2bc3-5b8d-4840-874b-f45c234e5b73`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:54:12,931::task::990::TaskManager.Task::(_decref) Task=`b40d2bc3-5b8d-4840-874b-f45c234e5b73`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:54:12,932::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:54:12,932::task::1185::TaskManager.Task::(prepare) Task=`b40d2bc3-5b8d-4840-874b-f45c234e5b73`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:54:12,933::task::595::TaskManager.Task::(_updateState) Task=`b40d2bc3-5b8d-4840-874b-f45c234e5b73`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:54:12,933::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:54:12,933::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:54:12,933::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:54:12,934::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:54:12,934::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:54:12,934::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:54:12,934::task::990::TaskManager.Task::(_decref) Task=`b40d2bc3-5b8d-4840-874b-f45c234e5b73`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:54:13,647::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:54:13,656::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000405802 s, 858 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:54:14,600::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:54:14,778::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:14,857::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0375377 s, 27.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:54:16,870::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:16,946::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313303 s, 32.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:54:17,622::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:54:18,420::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:54:18,429::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000762488 s, 5.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:54:18,664::task::595::TaskManager.Task::(_updateState) Task=`8e683d19-8fe5-42a6-a0a9-516e65b8e608`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:54:18,664::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:54:18,664::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000397805', 'lastCheck': '7.0', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000762488', 'lastCheck': '0.2', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000405802', 'lastCheck': '5.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:54:18,664::task::1185::TaskManager.Task::(prepare) Task=`8e683d19-8fe5-42a6-a0a9-516e65b8e608`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000397805', 'lastCheck': '7.0', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000762488', 'lastCheck': '0.2', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000405802', 'lastCheck': '5.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:54:18,665::task::595::TaskManager.Task::(_updateState) Task=`8e683d19-8fe5-42a6-a0a9-516e65b8e608`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:54:18,665::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:54:18,665::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:54:18,665::task::990::TaskManager.Task::(_decref) Task=`8e683d19-8fe5-42a6-a0a9-516e65b8e608`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:54:18,959::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:19,394::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.392383 s, 2.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:54:20,640::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:54:21,407::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:21,486::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361426 s, 28.3 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:54:21,626::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:54:21,636::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000760441 s, 514 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:54:22,979::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:54:22,979::task::595::TaskManager.Task::(_updateState) Task=`4f99af57-4861-4749-90fa-11d960595bf9`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:54:22,979::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:54:22,980::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:54:22,980::task::1185::TaskManager.Task::(prepare) Task=`4f99af57-4861-4749-90fa-11d960595bf9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:54:22,980::task::595::TaskManager.Task::(_updateState) Task=`4f99af57-4861-4749-90fa-11d960595bf9`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:54:22,980::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:54:22,980::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:54:22,980::task::990::TaskManager.Task::(_decref) Task=`4f99af57-4861-4749-90fa-11d960595bf9`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:54:22,985::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:54:22,985::task::595::TaskManager.Task::(_updateState) Task=`a6028145-ff38-40fb-b831-26b433ef91c9`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:54:22,985::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:54:22,986::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`bcc5eb7b-e44c-4450-9591-0e7420f82c40`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:54:22,986::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:54:22,986::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:54:22,986::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`bcc5eb7b-e44c-4450-9591-0e7420f82c40`::Granted request >Thread-338::DEBUG::2014-05-13 12:54:22,988::task::827::TaskManager.Task::(resourceAcquired) Task=`a6028145-ff38-40fb-b831-26b433ef91c9`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:54:22,988::task::990::TaskManager.Task::(_decref) Task=`a6028145-ff38-40fb-b831-26b433ef91c9`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:54:22,989::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:54:22,989::task::1185::TaskManager.Task::(prepare) Task=`a6028145-ff38-40fb-b831-26b433ef91c9`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:54:22,989::task::595::TaskManager.Task::(_updateState) Task=`a6028145-ff38-40fb-b831-26b433ef91c9`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:54:22,989::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:54:22,989::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:54:22,990::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:54:22,990::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:54:22,990::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:54:22,990::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:54:22,990::task::990::TaskManager.Task::(_decref) Task=`a6028145-ff38-40fb-b831-26b433ef91c9`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:54:23,499::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:23,573::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.030773 s, 33.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:54:23,661::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-344::DEBUG::2014-05-13 12:54:23,670::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:54:23,679::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000424946 s, 819 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:54:25,586::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:25,664::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0314616 s, 32.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:54:26,687::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:54:27,677::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:27,756::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0351914 s, 29.1 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:54:28,437::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:54:28,446::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000581619 s, 7.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:54:29,706::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:54:29,769::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:29,849::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363234 s, 28.2 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:54:31,647::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:54:31,657::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000355748 s, 1.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:54:31,862::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:31,944::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0375407 s, 27.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:54:32,724::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:54:33,028::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:54:33,029::task::595::TaskManager.Task::(_updateState) Task=`ab49c601-90f3-40c6-a9b2-4b5792695152`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:54:33,029::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:54:33,029::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:54:33,029::task::1185::TaskManager.Task::(prepare) Task=`ab49c601-90f3-40c6-a9b2-4b5792695152`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:54:33,030::task::595::TaskManager.Task::(_updateState) Task=`ab49c601-90f3-40c6-a9b2-4b5792695152`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:54:33,030::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:54:33,030::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:54:33,030::task::990::TaskManager.Task::(_decref) Task=`ab49c601-90f3-40c6-a9b2-4b5792695152`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:54:33,036::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:54:33,036::task::595::TaskManager.Task::(_updateState) Task=`416accc9-8ac5-4b35-98b3-d6d2ae996a69`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:54:33,036::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:54:33,037::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`15fccdbc-4209-4570-bb78-e012bd9b6fb5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:54:33,037::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:54:33,037::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:54:33,037::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`15fccdbc-4209-4570-bb78-e012bd9b6fb5`::Granted request >Thread-338::DEBUG::2014-05-13 12:54:33,038::task::827::TaskManager.Task::(resourceAcquired) Task=`416accc9-8ac5-4b35-98b3-d6d2ae996a69`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:54:33,038::task::990::TaskManager.Task::(_decref) Task=`416accc9-8ac5-4b35-98b3-d6d2ae996a69`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:54:33,039::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:54:33,039::task::1185::TaskManager.Task::(prepare) Task=`416accc9-8ac5-4b35-98b3-d6d2ae996a69`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:54:33,039::task::595::TaskManager.Task::(_updateState) Task=`416accc9-8ac5-4b35-98b3-d6d2ae996a69`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:54:33,040::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:54:33,040::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:54:33,040::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:54:33,040::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:54:33,041::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:54:33,041::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:54:33,041::task::990::TaskManager.Task::(_decref) Task=`416accc9-8ac5-4b35-98b3-d6d2ae996a69`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:54:33,690::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:54:33,699::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000479023 s, 726 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:54:33,805::task::595::TaskManager.Task::(_updateState) Task=`1a146719-1559-4841-9490-94f80b26ce42`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:54:33,805::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:54:33,805::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000355748', 'lastCheck': '2.1', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000581619', 'lastCheck': '5.4', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000479023', 'lastCheck': '0.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:54:33,806::task::1185::TaskManager.Task::(prepare) Task=`1a146719-1559-4841-9490-94f80b26ce42`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000355748', 'lastCheck': '2.1', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000581619', 'lastCheck': '5.4', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000479023', 'lastCheck': '0.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:54:33,806::task::595::TaskManager.Task::(_updateState) Task=`1a146719-1559-4841-9490-94f80b26ce42`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:54:33,806::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:54:33,806::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:54:33,806::task::990::TaskManager.Task::(_decref) Task=`1a146719-1559-4841-9490-94f80b26ce42`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:54:33,957::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:34,049::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0484196 s, 21.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:54:35,741::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:54:36,062::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:36,144::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0377688 s, 27.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:54:38,157::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:38,233::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315182 s, 32.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:54:38,454::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:54:38,462::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000635792 s, 6.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:54:38,760::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:54:40,246::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:40,327::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0366089 s, 28.0 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:54:41,668::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:54:41,678::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000932718 s, 419 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:54:41,779::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:54:42,340::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:42,419::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353344 s, 29.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:54:43,081::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:54:43,081::task::595::TaskManager.Task::(_updateState) Task=`2399e521-9124-41bc-8fa5-a12202ae75b5`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:54:43,081::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:54:43,082::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:54:43,082::task::1185::TaskManager.Task::(prepare) Task=`2399e521-9124-41bc-8fa5-a12202ae75b5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:54:43,082::task::595::TaskManager.Task::(_updateState) Task=`2399e521-9124-41bc-8fa5-a12202ae75b5`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:54:43,082::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:54:43,082::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:54:43,083::task::990::TaskManager.Task::(_decref) Task=`2399e521-9124-41bc-8fa5-a12202ae75b5`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:54:43,087::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:54:43,087::task::595::TaskManager.Task::(_updateState) Task=`16f8ad5e-4ddf-4467-ba02-a27220d86e6d`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:54:43,087::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:54:43,088::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`7ed4bec7-c7c3-4e5e-bb14-55968574ef4a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:54:43,088::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:54:43,088::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:54:43,089::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`7ed4bec7-c7c3-4e5e-bb14-55968574ef4a`::Granted request >Thread-338::DEBUG::2014-05-13 12:54:43,089::task::827::TaskManager.Task::(resourceAcquired) Task=`16f8ad5e-4ddf-4467-ba02-a27220d86e6d`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:54:43,089::task::990::TaskManager.Task::(_decref) Task=`16f8ad5e-4ddf-4467-ba02-a27220d86e6d`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:54:43,090::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:54:43,090::task::1185::TaskManager.Task::(prepare) Task=`16f8ad5e-4ddf-4467-ba02-a27220d86e6d`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:54:43,091::task::595::TaskManager.Task::(_updateState) Task=`16f8ad5e-4ddf-4467-ba02-a27220d86e6d`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:54:43,091::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:54:43,091::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:54:43,091::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:54:43,091::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:54:43,092::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:54:43,092::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:54:43,092::task::990::TaskManager.Task::(_decref) Task=`16f8ad5e-4ddf-4467-ba02-a27220d86e6d`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:54:43,708::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:54:43,717::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.00040321 s, 863 kB/s\n'; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:54:43,896::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [257ebcac] >Thread-338::DEBUG::2014-05-13 12:54:43,930::task::595::TaskManager.Task::(_updateState) Task=`23448bec-bb90-442a-b801-dbc25cf29ae1`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:54:43,930::logUtils::44::dispatcher::(wrapper) Run and protect: copyImage(sdUUID='572b0496-5eab-41cd-a976-4514f111898e', spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', vmUUID='', srcImgUUID='f35efc5c-cd67-4de9-b1c6-9e08bafcae46', srcVolUUID='c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a', dstImgUUID='f35efc5c-cd67-4de9-b1c6-9e08bafcae46', dstVolUUID='c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a', description='', dstSdUUID='41830dad-f012-4b4d-b636-863f66d5ca4b', volType=6, volFormat=5, preallocate=2, postZero='false', force='false') >Thread-338::DEBUG::2014-05-13 12:54:43,933::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a >Thread-338::INFO::2014-05-13 12:54:43,941::image::285::Storage.Image::(isLegal) image f35efc5c-cd67-4de9-b1c6-9e08bafcae46 in domain 572b0496-5eab-41cd-a976-4514f111898e has vollist ['c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a'] >Thread-338::DEBUG::2014-05-13 12:54:43,942::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a >Thread-338::DEBUG::2014-05-13 12:54:43,956::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a >Thread-338::DEBUG::2014-05-13 12:54:43,964::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a >Thread-338::ERROR::2014-05-13 12:54:43,978::task::866::TaskManager.Task::(_setError) Task=`23448bec-bb90-442a-b801-dbc25cf29ae1`::Unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/storage/task.py", line 873, in _run > return fn(*args, **kargs) > File "/usr/share/vdsm/logUtils.py", line 45, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/storage/hsm.py", line 1782, in copyImage > volFormat, preallocate, volume.BLANK_UUID) > File "/usr/share/vdsm/storage/blockSD.py", line 645, in validateCreateVolumeParams > volFormat, preallocate, srcVolUUID) > File "/usr/share/vdsm/storage/blockVolume.py", line 127, in validateCreateVolumeParams > raise se.IncorrectFormat(srcVolUUID) >IncorrectFormat: Incorrect Volume format: ('00000000-0000-0000-0000-000000000000',) >Thread-338::DEBUG::2014-05-13 12:54:44,002::task::885::TaskManager.Task::(_run) Task=`23448bec-bb90-442a-b801-dbc25cf29ae1`::Task._run: 23448bec-bb90-442a-b801-dbc25cf29ae1 ('572b0496-5eab-41cd-a976-4514f111898e', 'aa069361-8e06-4459-a17b-748c0a2a828e', '', 'f35efc5c-cd67-4de9-b1c6-9e08bafcae46', 'c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a', 'f35efc5c-cd67-4de9-b1c6-9e08bafcae46', 'c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a', '', '41830dad-f012-4b4d-b636-863f66d5ca4b', 6, 5, 2, 'false', 'false') {} failed - stopping task >Thread-338::DEBUG::2014-05-13 12:54:44,002::task::1211::TaskManager.Task::(stop) Task=`23448bec-bb90-442a-b801-dbc25cf29ae1`::stopping in state preparing (force False) >Thread-338::DEBUG::2014-05-13 12:54:44,002::task::990::TaskManager.Task::(_decref) Task=`23448bec-bb90-442a-b801-dbc25cf29ae1`::ref 1 aborting True >Thread-338::INFO::2014-05-13 12:54:44,002::task::1168::TaskManager.Task::(prepare) Task=`23448bec-bb90-442a-b801-dbc25cf29ae1`::aborting: Task is aborted: 'Incorrect Volume format' - code 202 >Thread-338::DEBUG::2014-05-13 12:54:44,002::task::1173::TaskManager.Task::(prepare) Task=`23448bec-bb90-442a-b801-dbc25cf29ae1`::Prepare: aborted: Incorrect Volume format >Thread-338::DEBUG::2014-05-13 12:54:44,003::task::990::TaskManager.Task::(_decref) Task=`23448bec-bb90-442a-b801-dbc25cf29ae1`::ref 0 aborting True >Thread-338::DEBUG::2014-05-13 12:54:44,003::task::925::TaskManager.Task::(_doAbort) Task=`23448bec-bb90-442a-b801-dbc25cf29ae1`::Task._doAbort: force False >Thread-338::DEBUG::2014-05-13 12:54:44,003::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:54:44,003::task::595::TaskManager.Task::(_updateState) Task=`23448bec-bb90-442a-b801-dbc25cf29ae1`::moving from state preparing -> state aborting >Thread-338::DEBUG::2014-05-13 12:54:44,003::task::550::TaskManager.Task::(__state_aborting) Task=`23448bec-bb90-442a-b801-dbc25cf29ae1`::_aborting: recover policy none >Thread-338::DEBUG::2014-05-13 12:54:44,003::task::595::TaskManager.Task::(_updateState) Task=`23448bec-bb90-442a-b801-dbc25cf29ae1`::moving from state aborting -> state failed >Thread-338::DEBUG::2014-05-13 12:54:44,004::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-338::DEBUG::2014-05-13 12:54:44,004::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::ERROR::2014-05-13 12:54:44,004::dispatcher::65::Storage.Dispatcher.Protect::(run) {'status': {'message': "Incorrect Volume format: ('00000000-0000-0000-0000-000000000000',)", 'code': 202}} >Dummy-337::DEBUG::2014-05-13 12:54:44,432::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:44,513::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0372941 s, 27.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:54:44,799::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:54:46,525::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:46,600::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0307045 s, 33.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:54:47,819::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:54:48,471::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:54:48,479::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000767359 s, 5.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:54:48,613::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:48,707::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0505155 s, 20.3 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:54:48,941::task::595::TaskManager.Task::(_updateState) Task=`69a79b69-ed50-49b9-a2c9-c215f9f8b23c`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:54:48,964::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:54:48,964::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000932718', 'lastCheck': '7.3', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000767359', 'lastCheck': '0.5', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00040321', 'lastCheck': '5.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:54:48,965::task::1185::TaskManager.Task::(prepare) Task=`69a79b69-ed50-49b9-a2c9-c215f9f8b23c`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000932718', 'lastCheck': '7.3', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000767359', 'lastCheck': '0.5', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00040321', 'lastCheck': '5.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:54:48,965::task::595::TaskManager.Task::(_updateState) Task=`69a79b69-ed50-49b9-a2c9-c215f9f8b23c`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:54:48,965::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:54:48,965::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:54:48,965::task::990::TaskManager.Task::(_decref) Task=`69a79b69-ed50-49b9-a2c9-c215f9f8b23c`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:54:50,719::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:50,819::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0564905 s, 18.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:54:50,837::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-357::DEBUG::2014-05-13 12:54:51,690::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:54:51,699::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000408738 s, 957 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:54:52,832::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:52,906::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0304955 s, 33.6 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:54:53,138::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4478bd69] >Thread-66::DEBUG::2014-05-13 12:54:53,138::task::595::TaskManager.Task::(_updateState) Task=`2cee0e42-1aad-4e87-91b5-84edd27dc193`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:54:53,139::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:54:53,139::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:54:53,139::task::1185::TaskManager.Task::(prepare) Task=`2cee0e42-1aad-4e87-91b5-84edd27dc193`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:54:53,139::task::595::TaskManager.Task::(_updateState) Task=`2cee0e42-1aad-4e87-91b5-84edd27dc193`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:54:53,140::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:54:53,140::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:54:53,140::task::990::TaskManager.Task::(_decref) Task=`2cee0e42-1aad-4e87-91b5-84edd27dc193`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:54:53,145::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [4478bd69] >Thread-338::DEBUG::2014-05-13 12:54:53,145::task::595::TaskManager.Task::(_updateState) Task=`c8909bdd-4af6-4cef-8290-673d09ece97b`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:54:53,145::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:54:53,146::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d68cf909-20e7-4fc3-bb3d-722fcc997354`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:54:53,146::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:54:53,146::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:54:53,147::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`d68cf909-20e7-4fc3-bb3d-722fcc997354`::Granted request >Thread-338::DEBUG::2014-05-13 12:54:53,147::task::827::TaskManager.Task::(resourceAcquired) Task=`c8909bdd-4af6-4cef-8290-673d09ece97b`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:54:53,147::task::990::TaskManager.Task::(_decref) Task=`c8909bdd-4af6-4cef-8290-673d09ece97b`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:54:53,148::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:54:53,148::task::1185::TaskManager.Task::(prepare) Task=`c8909bdd-4af6-4cef-8290-673d09ece97b`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:54:53,149::task::595::TaskManager.Task::(_updateState) Task=`c8909bdd-4af6-4cef-8290-673d09ece97b`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:54:53,149::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:54:53,149::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:54:53,149::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:54:53,149::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:54:53,150::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:54:53,150::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:54:53,150::task::990::TaskManager.Task::(_decref) Task=`c8909bdd-4af6-4cef-8290-673d09ece97b`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:54:53,728::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:54:53,737::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000481162 s, 723 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:54:53,855::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:54:54,919::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:55,006::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0428515 s, 23.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:54:56,872::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:54:57,019::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:57,100::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0382155 s, 26.8 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:54:58,487::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:54:58,498::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00235026 s, 1.7 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:54:59,113::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:54:59,193::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0378614 s, 27.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:54:59,890::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:55:01,206::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:55:01,291::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0425188 s, 24.1 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:55:01,712::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:55:01,723::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000448405 s, 872 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:55:02,907::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:55:03,185::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:55:03,185::task::595::TaskManager.Task::(_updateState) Task=`841acd45-6656-4295-9a7c-f94b425532a8`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:55:03,186::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:55:03,186::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:55:03,186::task::1185::TaskManager.Task::(prepare) Task=`841acd45-6656-4295-9a7c-f94b425532a8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:55:03,186::task::595::TaskManager.Task::(_updateState) Task=`841acd45-6656-4295-9a7c-f94b425532a8`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:55:03,186::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:55:03,187::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:55:03,187::task::990::TaskManager.Task::(_decref) Task=`841acd45-6656-4295-9a7c-f94b425532a8`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:55:03,192::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:55:03,192::task::595::TaskManager.Task::(_updateState) Task=`f4e46822-5a14-44cc-8b23-d3f4b59cb557`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:55:03,192::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:55:03,193::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`96276c3a-bce1-4405-b5c5-decac38a330c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:55:03,193::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:55:03,193::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:55:03,193::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`96276c3a-bce1-4405-b5c5-decac38a330c`::Granted request >Thread-338::DEBUG::2014-05-13 12:55:03,194::task::827::TaskManager.Task::(resourceAcquired) Task=`f4e46822-5a14-44cc-8b23-d3f4b59cb557`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:55:03,194::task::990::TaskManager.Task::(_decref) Task=`f4e46822-5a14-44cc-8b23-d3f4b59cb557`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:55:03,195::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:55:03,195::task::1185::TaskManager.Task::(prepare) Task=`f4e46822-5a14-44cc-8b23-d3f4b59cb557`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:55:03,196::task::595::TaskManager.Task::(_updateState) Task=`f4e46822-5a14-44cc-8b23-d3f4b59cb557`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:55:03,196::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:55:03,196::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:55:03,196::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:55:03,196::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:55:03,197::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:55:03,197::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:55:03,197::task::990::TaskManager.Task::(_decref) Task=`f4e46822-5a14-44cc-8b23-d3f4b59cb557`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:55:03,304::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:55:03,385::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383553 s, 26.7 MB/s\n'; <rc> = 0 >Thread-344::DEBUG::2014-05-13 12:55:03,749::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:55:03,758::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000411399 s, 846 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:55:04,106::task::595::TaskManager.Task::(_updateState) Task=`2086af64-41d9-44d3-927d-1e1538f57372`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:55:04,106::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:55:04,107::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000448405', 'lastCheck': '2.4', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00235026', 'lastCheck': '5.6', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000411399', 'lastCheck': '0.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:55:04,107::task::1185::TaskManager.Task::(prepare) Task=`2086af64-41d9-44d3-927d-1e1538f57372`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000448405', 'lastCheck': '2.4', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00235026', 'lastCheck': '5.6', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000411399', 'lastCheck': '0.3', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:55:04,107::task::595::TaskManager.Task::(_updateState) Task=`2086af64-41d9-44d3-927d-1e1538f57372`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:55:04,107::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:55:04,107::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:55:04,107::task::990::TaskManager.Task::(_decref) Task=`2086af64-41d9-44d3-927d-1e1538f57372`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:55:05,398::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:55:05,478::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374443 s, 27.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:55:05,927::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:55:07,491::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:55:07,566::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0335354 s, 30.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:55:08,506::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:55:08,514::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000620578 s, 6.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:55:08,948::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:55:09,579::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:55:09,658::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355594 s, 28.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:55:11,671::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-357::DEBUG::2014-05-13 12:55:11,786::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:55:11,805::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000306981 s, 1.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:55:11,830::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0376685 s, 27.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:55:11,967::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:55:13,233::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [303c64a0] >Thread-66::DEBUG::2014-05-13 12:55:13,233::task::595::TaskManager.Task::(_updateState) Task=`ccafa240-c354-484a-ad21-702a1d3b8a46`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:55:13,234::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:55:13,234::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:55:13,234::task::1185::TaskManager.Task::(prepare) Task=`ccafa240-c354-484a-ad21-702a1d3b8a46`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:55:13,234::task::595::TaskManager.Task::(_updateState) Task=`ccafa240-c354-484a-ad21-702a1d3b8a46`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:55:13,235::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:55:13,235::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:55:13,235::task::990::TaskManager.Task::(_decref) Task=`ccafa240-c354-484a-ad21-702a1d3b8a46`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:55:13,279::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [303c64a0] >Thread-338::DEBUG::2014-05-13 12:55:13,280::task::595::TaskManager.Task::(_updateState) Task=`1383ace6-3eee-4462-9846-ff9ed6ff74f2`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:55:13,280::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:55:13,280::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0c07c78b-ba93-4c01-bc49-d427158f6d70`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:55:13,280::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:55:13,281::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:55:13,281::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`0c07c78b-ba93-4c01-bc49-d427158f6d70`::Granted request >Thread-338::DEBUG::2014-05-13 12:55:13,281::task::827::TaskManager.Task::(resourceAcquired) Task=`1383ace6-3eee-4462-9846-ff9ed6ff74f2`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:55:13,281::task::990::TaskManager.Task::(_decref) Task=`1383ace6-3eee-4462-9846-ff9ed6ff74f2`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:55:13,283::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:55:13,283::task::1185::TaskManager.Task::(prepare) Task=`1383ace6-3eee-4462-9846-ff9ed6ff74f2`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:55:13,283::task::595::TaskManager.Task::(_updateState) Task=`1383ace6-3eee-4462-9846-ff9ed6ff74f2`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:55:13,283::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:55:13,284::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:55:13,284::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:55:13,284::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:55:13,284::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:55:13,284::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:55:13,285::task::990::TaskManager.Task::(_decref) Task=`1383ace6-3eee-4462-9846-ff9ed6ff74f2`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:55:13,774::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:55:13,784::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000432332 s, 805 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:55:13,843::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:55:14,257::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.372898 s, 2.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:55:14,987::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:55:16,270::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:55:16,343::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303119 s, 33.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:55:18,012::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:55:18,356::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:55:18,435::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356722 s, 28.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:55:18,520::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 41830dad-f012-4b4d-b636-863f66d5ca4b >Thread-322::DEBUG::2014-05-13 12:55:18,520::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-322::DEBUG::2014-05-13 12:55:18,523::blockSD::336::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:55:18,532::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000622652 s, 3.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:55:18,532::misc::287::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000622652 s, 3.3 MB/s'], size: 2048 >Thread-322::DEBUG::2014-05-13 12:55:18,532::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-322::WARNING::2014-05-13 12:55:18,533::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-322::DEBUG::2014-05-13 12:55:18,533::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-322::DEBUG::2014-05-13 12:55:18,533::lvm::503::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:55:18,534::lvm::505::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:55:18,534::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:55:18,534::lvm::525::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:55:18,535::lvm::373::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-322::DEBUG::2014-05-13 12:55:18,535::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000206625b2cfb5e311|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000246625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 12:55:18,860::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:55:18,862::lvm::415::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-322::DEBUG::2014-05-13 12:55:18,862::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'LOCKPOLICY=', 'VERSION=3', 'VGUUID=p5z3JE-4jx4-yzvF-hpFg-6Pvi-7rHk-UahOxO', 'DESCRIPTION=ISCSI', 'LOGBLKSIZE=512', 'TYPE=ISCSI', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'PHYBLKSIZE=512', u'PV2=pv:36006016012823000226625b2cfb5e311,uuid:DPT6Y0-g8qs-ybQW-Tmqv-ZXJN-JjZ4-PLdIpG,pestart:0,pecount:397,mapoffset:794', u'PV0=pv:36006016012823000266625b2cfb5e311,uuid:u18LmV-dS01-YKqL-6OL3-z3qg-6MuL-f20H1K,pestart:0,pecount:397,mapoffset:0', u'PV1=pv:36006016012823000286625b2cfb5e311,uuid:Hi4N3p-9yS4-CBqd-bRVe-YhKP-MnoD-DZHAx2,pestart:0,pecount:397,mapoffset:397', 'CLASS=Data', 'SDUUID=41830dad-f012-4b4d-b636-863f66d5ca4b', 'POOL_DESCRIPTION=DC_2', 'MASTER_VERSION=1', 'ROLE=Master', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', u'POOL_DOMAINS=208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', '_SHA_CKSUM=985bc0a261dab0d9f33927b045f395c62a74e955'] >Thread-322::INFO::2014-05-13 12:55:18,864::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_imageNS already registered >Thread-322::INFO::2014-05-13 12:55:18,864::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_volumeNS already registered >Thread-322::INFO::2014-05-13 12:55:18,864::blockSD::452::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 41830dad-f012-4b4d-b636-863f66d5ca4b_lvmActivationNS already registered >Thread-322::DEBUG::2014-05-13 12:55:18,865::lvm::296::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'a|/dev/mapper/36006016012823000226625b2cfb5e311|/dev/mapper/36006016012823000266625b2cfb5e311|/dev/mapper/36006016012823000286625b2cfb5e311|\', \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } " 41830dad-f012-4b4d-b636-863f66d5ca4b' (cwd None) >Thread-322::DEBUG::2014-05-13 12:55:19,078::lvm::296::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:55:19,083::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:55:19,092::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000701431 s, 5.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:55:19,238::task::595::TaskManager.Task::(_updateState) Task=`8a928c3f-442e-4f81-9c5a-f5eae303a896`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:55:19,239::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:55:19,239::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000306981', 'lastCheck': '7.4', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000701431', 'lastCheck': '0.1', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000432332', 'lastCheck': '5.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:55:19,239::task::1185::TaskManager.Task::(prepare) Task=`8a928c3f-442e-4f81-9c5a-f5eae303a896`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000306981', 'lastCheck': '7.4', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000701431', 'lastCheck': '0.1', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000432332', 'lastCheck': '5.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:55:19,239::task::595::TaskManager.Task::(_updateState) Task=`8a928c3f-442e-4f81-9c5a-f5eae303a896`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:55:19,239::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:55:19,239::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:55:19,240::task::990::TaskManager.Task::(_decref) Task=`8a928c3f-442e-4f81-9c5a-f5eae303a896`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:55:20,449::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:55:20,527::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353429 s, 29.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:55:21,036::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-357::DEBUG::2014-05-13 12:55:21,824::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:55:21,833::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000552722 s, 707 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:55:22,540::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:55:22,613::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0304016 s, 33.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:55:23,319::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:55:23,352::task::595::TaskManager.Task::(_updateState) Task=`fefca8ae-e334-4aa4-b1a2-25ab4c829eee`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:55:23,352::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:55:23,353::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:55:23,353::task::1185::TaskManager.Task::(prepare) Task=`fefca8ae-e334-4aa4-b1a2-25ab4c829eee`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:55:23,353::task::595::TaskManager.Task::(_updateState) Task=`fefca8ae-e334-4aa4-b1a2-25ab4c829eee`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:55:23,353::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:55:23,354::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:55:23,354::task::990::TaskManager.Task::(_decref) Task=`fefca8ae-e334-4aa4-b1a2-25ab4c829eee`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:55:23,358::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:55:23,359::task::595::TaskManager.Task::(_updateState) Task=`4025da0d-49eb-4668-ac60-bcf532f70464`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:55:23,359::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:55:23,359::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fd9e83d2-5852-48ed-a766-3dc8ca52c336`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:55:23,359::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:55:23,360::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:55:23,360::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`fd9e83d2-5852-48ed-a766-3dc8ca52c336`::Granted request >Thread-338::DEBUG::2014-05-13 12:55:23,360::task::827::TaskManager.Task::(resourceAcquired) Task=`4025da0d-49eb-4668-ac60-bcf532f70464`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:55:23,361::task::990::TaskManager.Task::(_decref) Task=`4025da0d-49eb-4668-ac60-bcf532f70464`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:55:23,361::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:55:23,362::task::1185::TaskManager.Task::(prepare) Task=`4025da0d-49eb-4668-ac60-bcf532f70464`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:55:23,362::task::595::TaskManager.Task::(_updateState) Task=`4025da0d-49eb-4668-ac60-bcf532f70464`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:55:23,362::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:55:23,362::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:55:23,363::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:55:23,363::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:55:23,363::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:55:23,363::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:55:23,364::task::990::TaskManager.Task::(_decref) Task=`4025da0d-49eb-4668-ac60-bcf532f70464`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:55:23,795::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:55:23,804::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000442139 s, 787 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:55:24,062::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:55:24,626::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:55:24,707::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037917 s, 27.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:55:26,720::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:55:26,799::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0351877 s, 29.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:55:27,081::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:55:28,811::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:55:28,921::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0453341 s, 22.6 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:55:29,101::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:55:29,109::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000586638 s, 7.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:55:30,108::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-338::DEBUG::2014-05-13 12:55:30,786::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [74f1053e] >Thread-338::DEBUG::2014-05-13 12:55:30,786::task::595::TaskManager.Task::(_updateState) Task=`7332cfee-9406-4f69-8745-01f6a9646e12`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:55:30,787::logUtils::44::dispatcher::(wrapper) Run and protect: copyImage(sdUUID='572b0496-5eab-41cd-a976-4514f111898e', spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', vmUUID='', srcImgUUID='f35efc5c-cd67-4de9-b1c6-9e08bafcae46', srcVolUUID='c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a', dstImgUUID='f35efc5c-cd67-4de9-b1c6-9e08bafcae46', dstVolUUID='c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a', description='', dstSdUUID='41830dad-f012-4b4d-b636-863f66d5ca4b', volType=6, volFormat=5, preallocate=2, postZero='false', force='false') >Thread-338::DEBUG::2014-05-13 12:55:30,789::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a >Thread-338::INFO::2014-05-13 12:55:30,797::image::285::Storage.Image::(isLegal) image f35efc5c-cd67-4de9-b1c6-9e08bafcae46 in domain 572b0496-5eab-41cd-a976-4514f111898e has vollist ['c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a'] >Thread-338::DEBUG::2014-05-13 12:55:30,798::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a >Thread-338::DEBUG::2014-05-13 12:55:30,812::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a >Thread-338::DEBUG::2014-05-13 12:55:30,820::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a >Thread-338::ERROR::2014-05-13 12:55:30,833::task::866::TaskManager.Task::(_setError) Task=`7332cfee-9406-4f69-8745-01f6a9646e12`::Unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/storage/task.py", line 873, in _run > return fn(*args, **kargs) > File "/usr/share/vdsm/logUtils.py", line 45, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/storage/hsm.py", line 1782, in copyImage > volFormat, preallocate, volume.BLANK_UUID) > File "/usr/share/vdsm/storage/blockSD.py", line 645, in validateCreateVolumeParams > volFormat, preallocate, srcVolUUID) > File "/usr/share/vdsm/storage/blockVolume.py", line 127, in validateCreateVolumeParams > raise se.IncorrectFormat(srcVolUUID) >IncorrectFormat: Incorrect Volume format: ('00000000-0000-0000-0000-000000000000',) >Thread-338::DEBUG::2014-05-13 12:55:30,833::task::885::TaskManager.Task::(_run) Task=`7332cfee-9406-4f69-8745-01f6a9646e12`::Task._run: 7332cfee-9406-4f69-8745-01f6a9646e12 ('572b0496-5eab-41cd-a976-4514f111898e', 'aa069361-8e06-4459-a17b-748c0a2a828e', '', 'f35efc5c-cd67-4de9-b1c6-9e08bafcae46', 'c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a', 'f35efc5c-cd67-4de9-b1c6-9e08bafcae46', 'c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a', '', '41830dad-f012-4b4d-b636-863f66d5ca4b', 6, 5, 2, 'false', 'false') {} failed - stopping task >Thread-338::DEBUG::2014-05-13 12:55:30,834::task::1211::TaskManager.Task::(stop) Task=`7332cfee-9406-4f69-8745-01f6a9646e12`::stopping in state preparing (force False) >Thread-338::DEBUG::2014-05-13 12:55:30,834::task::990::TaskManager.Task::(_decref) Task=`7332cfee-9406-4f69-8745-01f6a9646e12`::ref 1 aborting True >Thread-338::INFO::2014-05-13 12:55:30,834::task::1168::TaskManager.Task::(prepare) Task=`7332cfee-9406-4f69-8745-01f6a9646e12`::aborting: Task is aborted: 'Incorrect Volume format' - code 202 >Thread-338::DEBUG::2014-05-13 12:55:30,834::task::1173::TaskManager.Task::(prepare) Task=`7332cfee-9406-4f69-8745-01f6a9646e12`::Prepare: aborted: Incorrect Volume format >Thread-338::DEBUG::2014-05-13 12:55:30,835::task::990::TaskManager.Task::(_decref) Task=`7332cfee-9406-4f69-8745-01f6a9646e12`::ref 0 aborting True >Thread-338::DEBUG::2014-05-13 12:55:30,835::task::925::TaskManager.Task::(_doAbort) Task=`7332cfee-9406-4f69-8745-01f6a9646e12`::Task._doAbort: force False >Thread-338::DEBUG::2014-05-13 12:55:30,835::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:55:30,835::task::595::TaskManager.Task::(_updateState) Task=`7332cfee-9406-4f69-8745-01f6a9646e12`::moving from state preparing -> state aborting >Thread-338::DEBUG::2014-05-13 12:55:30,835::task::550::TaskManager.Task::(__state_aborting) Task=`7332cfee-9406-4f69-8745-01f6a9646e12`::_aborting: recover policy none >Thread-338::DEBUG::2014-05-13 12:55:30,836::task::595::TaskManager.Task::(_updateState) Task=`7332cfee-9406-4f69-8745-01f6a9646e12`::moving from state aborting -> state failed >Thread-338::DEBUG::2014-05-13 12:55:30,836::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-338::DEBUG::2014-05-13 12:55:30,836::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::ERROR::2014-05-13 12:55:30,836::dispatcher::65::Storage.Dispatcher.Protect::(run) {'status': {'message': "Incorrect Volume format: ('00000000-0000-0000-0000-000000000000',)", 'code': 202}} >Dummy-337::DEBUG::2014-05-13 12:55:30,935::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:55:31,017::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0372909 s, 27.5 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:55:31,843::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:55:31,853::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000465504 s, 840 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:55:33,030::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:55:33,113::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0388791 s, 26.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:55:33,128::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:55:33,402::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3179350e] >Thread-66::DEBUG::2014-05-13 12:55:33,402::task::595::TaskManager.Task::(_updateState) Task=`89d5af59-ad77-460b-8e97-14a77e83f112`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:55:33,402::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:55:33,403::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:55:33,403::task::1185::TaskManager.Task::(prepare) Task=`89d5af59-ad77-460b-8e97-14a77e83f112`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:55:33,403::task::595::TaskManager.Task::(_updateState) Task=`89d5af59-ad77-460b-8e97-14a77e83f112`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:55:33,403::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:55:33,403::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:55:33,404::task::990::TaskManager.Task::(_decref) Task=`89d5af59-ad77-460b-8e97-14a77e83f112`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:55:33,409::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [3179350e] >Thread-338::DEBUG::2014-05-13 12:55:33,409::task::595::TaskManager.Task::(_updateState) Task=`d8f5506c-64d6-400f-9e1b-fe589d793ab1`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:55:33,409::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:55:33,410::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`49cb3a88-06e4-45ef-83da-ba033b939952`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:55:33,410::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:55:33,410::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:55:33,410::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`49cb3a88-06e4-45ef-83da-ba033b939952`::Granted request >Thread-338::DEBUG::2014-05-13 12:55:33,411::task::827::TaskManager.Task::(resourceAcquired) Task=`d8f5506c-64d6-400f-9e1b-fe589d793ab1`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:55:33,411::task::990::TaskManager.Task::(_decref) Task=`d8f5506c-64d6-400f-9e1b-fe589d793ab1`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:55:33,412::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:55:33,412::task::1185::TaskManager.Task::(prepare) Task=`d8f5506c-64d6-400f-9e1b-fe589d793ab1`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:55:33,412::task::595::TaskManager.Task::(_updateState) Task=`d8f5506c-64d6-400f-9e1b-fe589d793ab1`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:55:33,412::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:55:33,413::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:55:33,413::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:55:33,413::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:55:33,413::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:55:33,413::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:55:33,414::task::990::TaskManager.Task::(_decref) Task=`d8f5506c-64d6-400f-9e1b-fe589d793ab1`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:55:33,815::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:55:33,825::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000474068 s, 734 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:55:34,373::task::595::TaskManager.Task::(_updateState) Task=`33f3d79d-34f3-4032-b045-3d3db9c2230a`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:55:34,373::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:55:34,373::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000465504', 'lastCheck': '2.5', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000586638', 'lastCheck': '5.3', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000474068', 'lastCheck': '0.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:55:34,373::task::1185::TaskManager.Task::(prepare) Task=`33f3d79d-34f3-4032-b045-3d3db9c2230a`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000465504', 'lastCheck': '2.5', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000586638', 'lastCheck': '5.3', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000474068', 'lastCheck': '0.5', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:55:34,374::task::595::TaskManager.Task::(_updateState) Task=`33f3d79d-34f3-4032-b045-3d3db9c2230a`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:55:34,374::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:55:34,374::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:55:34,374::task::990::TaskManager.Task::(_decref) Task=`33f3d79d-34f3-4032-b045-3d3db9c2230a`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:55:35,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:55:35,203::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316456 s, 32.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:55:36,147::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:55:37,216::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:55:37,293::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031983 s, 32.0 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:55:39,118::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:55:39,126::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000666879 s, 6.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:55:39,165::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:55:39,306::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:55:39,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374095 s, 27.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:55:41,400::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:55:41,817::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.373574 s, 2.7 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:55:41,864::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:55:41,873::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000653887 s, 598 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:55:42,184::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:55:43,468::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:55:43,468::task::595::TaskManager.Task::(_updateState) Task=`8a7edb5c-7207-4000-9f82-8d4e83110a73`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:55:43,469::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:55:43,469::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:55:43,469::task::1185::TaskManager.Task::(prepare) Task=`8a7edb5c-7207-4000-9f82-8d4e83110a73`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:55:43,469::task::595::TaskManager.Task::(_updateState) Task=`8a7edb5c-7207-4000-9f82-8d4e83110a73`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:55:43,469::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:55:43,469::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:55:43,470::task::990::TaskManager.Task::(_decref) Task=`8a7edb5c-7207-4000-9f82-8d4e83110a73`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:55:43,474::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:55:43,474::task::595::TaskManager.Task::(_updateState) Task=`ebace27a-b9af-402c-b051-93a2c576dba7`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:55:43,475::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:55:43,475::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e5f58e90-6261-456b-9107-abe04b369324`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:55:43,475::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:55:43,475::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:55:43,476::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e5f58e90-6261-456b-9107-abe04b369324`::Granted request >Thread-338::DEBUG::2014-05-13 12:55:43,476::task::827::TaskManager.Task::(resourceAcquired) Task=`ebace27a-b9af-402c-b051-93a2c576dba7`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:55:43,476::task::990::TaskManager.Task::(_decref) Task=`ebace27a-b9af-402c-b051-93a2c576dba7`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:55:43,477::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:55:43,477::task::1185::TaskManager.Task::(prepare) Task=`ebace27a-b9af-402c-b051-93a2c576dba7`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:55:43,477::task::595::TaskManager.Task::(_updateState) Task=`ebace27a-b9af-402c-b051-93a2c576dba7`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:55:43,477::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:55:43,478::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:55:43,478::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:55:43,478::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:55:43,478::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:55:43,478::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:55:43,479::task::990::TaskManager.Task::(_decref) Task=`ebace27a-b9af-402c-b051-93a2c576dba7`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:55:43,830::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-344::DEBUG::2014-05-13 12:55:43,843::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:55:43,861::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000357041 s, 975 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:55:43,929::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0563721 s, 18.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:55:45,203::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:55:45,942::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:55:46,022::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0363276 s, 28.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:55:48,036::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:55:48,109::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.03036 s, 33.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:55:48,220::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:55:49,164::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:55:49,172::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00071939 s, 5.7 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:55:49,552::task::595::TaskManager.Task::(_updateState) Task=`6f5371c8-5199-40e9-a589-32e7f41eed60`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:55:49,552::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:55:49,553::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000653887', 'lastCheck': '7.7', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00071939', 'lastCheck': '0.4', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000357041', 'lastCheck': '5.7', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:55:49,553::task::1185::TaskManager.Task::(prepare) Task=`6f5371c8-5199-40e9-a589-32e7f41eed60`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000653887', 'lastCheck': '7.7', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00071939', 'lastCheck': '0.4', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000357041', 'lastCheck': '5.7', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:55:49,553::task::595::TaskManager.Task::(_updateState) Task=`6f5371c8-5199-40e9-a589-32e7f41eed60`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:55:49,553::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:55:49,553::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:55:49,553::task::990::TaskManager.Task::(_decref) Task=`6f5371c8-5199-40e9-a589-32e7f41eed60`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:55:50,122::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:55:50,198::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0337806 s, 30.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:55:51,244::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-357::DEBUG::2014-05-13 12:55:51,885::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:55:51,895::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000679221 s, 576 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:55:52,211::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:55:52,287::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0330354 s, 31.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:55:53,521::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:55:53,521::task::595::TaskManager.Task::(_updateState) Task=`fd365695-fa61-4b65-a312-7905f501e433`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:55:53,522::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:55:53,522::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:55:53,522::task::1185::TaskManager.Task::(prepare) Task=`fd365695-fa61-4b65-a312-7905f501e433`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:55:53,522::task::595::TaskManager.Task::(_updateState) Task=`fd365695-fa61-4b65-a312-7905f501e433`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:55:53,523::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:55:53,523::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:55:53,523::task::990::TaskManager.Task::(_decref) Task=`fd365695-fa61-4b65-a312-7905f501e433`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:55:53,529::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:55:53,529::task::595::TaskManager.Task::(_updateState) Task=`36038cda-2994-4ac8-8121-d91754d89912`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:55:53,530::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:55:53,530::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e8588533-bf19-464b-9b16-fa983254d979`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:55:53,530::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:55:53,531::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:55:53,531::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`e8588533-bf19-464b-9b16-fa983254d979`::Granted request >Thread-338::DEBUG::2014-05-13 12:55:53,531::task::827::TaskManager.Task::(resourceAcquired) Task=`36038cda-2994-4ac8-8121-d91754d89912`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:55:53,531::task::990::TaskManager.Task::(_decref) Task=`36038cda-2994-4ac8-8121-d91754d89912`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:55:53,532::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:55:53,532::task::1185::TaskManager.Task::(prepare) Task=`36038cda-2994-4ac8-8121-d91754d89912`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:55:53,533::task::595::TaskManager.Task::(_updateState) Task=`36038cda-2994-4ac8-8121-d91754d89912`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:55:53,533::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:55:53,533::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:55:53,533::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:55:53,534::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:55:53,534::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:55:53,534::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:55:53,534::task::990::TaskManager.Task::(_decref) Task=`36038cda-2994-4ac8-8121-d91754d89912`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:55:53,877::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:55:53,886::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000474183 s, 734 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:55:54,263::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:55:54,300::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:55:54,392::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0492954 s, 20.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:55:56,405::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:55:56,485::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0355306 s, 28.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:55:57,283::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:55:58,498::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:55:58,572::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.030787 s, 33.3 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:55:59,181::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:55:59,189::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000715521 s, 5.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:56:00,301::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:56:00,585::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:56:00,665::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0378356 s, 27.1 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:56:01,906::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:56:01,916::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000738962 s, 529 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:56:02,678::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:56:02,758::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381817 s, 26.8 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:56:03,321::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:56:03,591::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:56:03,591::task::595::TaskManager.Task::(_updateState) Task=`413e0191-10a8-4b83-90e8-766a84148c64`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:56:03,591::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:56:03,592::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:56:03,592::task::1185::TaskManager.Task::(prepare) Task=`413e0191-10a8-4b83-90e8-766a84148c64`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:56:03,592::task::595::TaskManager.Task::(_updateState) Task=`413e0191-10a8-4b83-90e8-766a84148c64`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:56:03,592::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:56:03,593::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:56:03,593::task::990::TaskManager.Task::(_decref) Task=`413e0191-10a8-4b83-90e8-766a84148c64`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:56:03,597::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:56:03,598::task::595::TaskManager.Task::(_updateState) Task=`a4ab8d87-d4c7-4801-a9da-797e85bdd697`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:56:03,598::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:56:03,598::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4335db6b-e31c-44c0-bfd0-f62614e43d0c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:56:03,598::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:56:03,599::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:56:03,599::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4335db6b-e31c-44c0-bfd0-f62614e43d0c`::Granted request >Thread-338::DEBUG::2014-05-13 12:56:03,599::task::827::TaskManager.Task::(resourceAcquired) Task=`a4ab8d87-d4c7-4801-a9da-797e85bdd697`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:56:03,600::task::990::TaskManager.Task::(_decref) Task=`a4ab8d87-d4c7-4801-a9da-797e85bdd697`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:56:03,600::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:56:03,601::task::1185::TaskManager.Task::(prepare) Task=`a4ab8d87-d4c7-4801-a9da-797e85bdd697`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:56:03,601::task::595::TaskManager.Task::(_updateState) Task=`a4ab8d87-d4c7-4801-a9da-797e85bdd697`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:56:03,601::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:56:03,601::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:56:03,602::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:56:03,602::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:56:03,602::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:56:03,602::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:56:03,602::task::990::TaskManager.Task::(_decref) Task=`a4ab8d87-d4c7-4801-a9da-797e85bdd697`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:56:03,898::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:56:03,908::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000855754 s, 407 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:56:04,687::task::595::TaskManager.Task::(_updateState) Task=`273b4b8b-4829-4179-88ce-1a5c6de11920`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:56:04,687::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:56:04,687::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000738962', 'lastCheck': '2.8', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000715521', 'lastCheck': '5.5', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000855754', 'lastCheck': '0.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:56:04,687::task::1185::TaskManager.Task::(prepare) Task=`273b4b8b-4829-4179-88ce-1a5c6de11920`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000738962', 'lastCheck': '2.8', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000715521', 'lastCheck': '5.5', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000855754', 'lastCheck': '0.8', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:56:04,688::task::595::TaskManager.Task::(_updateState) Task=`273b4b8b-4829-4179-88ce-1a5c6de11920`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:56:04,688::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:56:04,688::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:56:04,688::task::990::TaskManager.Task::(_decref) Task=`273b4b8b-4829-4179-88ce-1a5c6de11920`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:56:04,771::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:56:04,849::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0331305 s, 30.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:56:06,339::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:56:06,862::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:56:07,310::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.405554 s, 2.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:56:09,198::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:56:09,206::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000575257 s, 7.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:56:09,323::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:56:09,360::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:56:09,406::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0383245 s, 26.7 MB/s\n'; <rc> = 0 >Thread-338::DEBUG::2014-05-13 12:56:09,917::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:56:09,918::task::595::TaskManager.Task::(_updateState) Task=`4f0b806b-e21e-4979-bf22-68d3ac20cdcd`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:56:09,918::logUtils::44::dispatcher::(wrapper) Run and protect: updateVM(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', vmList=[{'imglist': 'ac82bfa3-8cd5-4643-b36a-5ad28040fe0a', 'ovf': '<?xml version=\'1.0\' encoding=\'UTF-8\'?><ovf:Envelope xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1/" xmlns:rasd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData" xmlns:vssd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" ovf:version="3.4.0.0"><References><File ovf:href="ac82bfa3-8cd5-4643-b36a-5ad28040fe0a/b4c44053-1991-4c45-aea6-2bcb369e4b65" ovf:id="b4c44053-1991-4c45-aea6-2bcb369e4b65" ovf:size="5368709120" ovf:description="Active VM"/></References><Section xsi:type="ovf:NetworkSection_Type"><Info>List of networks</Info><Network ovf:name="Network 1"/></Section><Section xsi:type="ovf:DiskSection_Type"><Info>List of Virtual Disks</Info><Disk ovf:diskId="b4c44053-1991-4c45-aea6-2bcb369e4b65" ovf:size="5" ovf:actual_size="0" ovf:vm_snapshot_id="d5b238c2-adea-45df-9213-4f80c6c6400f" ovf:parentRef="" ovf:fileRef="ac82bfa3-8cd5-4643-b36a-5ad28040fe0a/b4c44053-1991-4c45-aea6-2bcb369e4b65" ovf:format="http://www.vmware.com/specifications/vmdk.html#sparse" ovf:volume-format="RAW" ovf:volume-type="Sparse" ovf:disk-interface="VirtIO_SCSI" ovf:boot="true" ovf:disk-alias="vm1_Disk1" ovf:wipe-after-delete="false"/></Section><Content ovf:id="out" xsi:type="ovf:VirtualSystem_Type"><Description></Description><CreationDate>2014/05/13 09:38:17</CreationDate><ExportDate>2014/05/13 09:56:08</ExportDate><DeleteProtected>false</DeleteProtected><SsoMethod>guest_agent</SsoMethod><IsSmartcardEnabled>false</IsSmartcardEnabled><TimeZone></TimeZone><default_boot_sequence>0</default_boot_sequence><Generation>2</Generation><VmType>1</VmType><MinAllocatedMem>1024</MinAllocatedMem><IsStateless>false</IsStateless><IsRunAndPause>false</IsRunAndPause><AutoStartup>false</AutoStartup><Priority>1</Priority><CreatedByUserId>fdfc627c-d875-11e0-90f0-83df133b58cc</CreatedByUserId><Name>vm1</Name><TemplateId>00000000-0000-0000-0000-000000000000</TemplateId><TemplateName>Blank</TemplateName><IsInitilized>true</IsInitilized><Origin>3</Origin><DefaultDisplayType>1</DefaultDisplayType><TrustedService>false</TrustedService><OriginalTemplateId>00000000-0000-0000-0000-000000000000</OriginalTemplateId><OriginalTemplateName>Blank</OriginalTemplateName><UseLatestVersion>false</UseLatestVersion><Section ovf:id="4da6d3ed-e2fd-4318-a58f-715cc1756430" ovf:required="false" xsi:type="ovf:OperatingSystemSection_Type"><Info>Guest Operating System</Info><Description>other</Description></Section><Section xsi:type="ovf:VirtualHardwareSection_Type"><Info>1 CPU, 1024 Memeory</Info><System><vssd:VirtualSystemType>ENGINE 3.4.0.0</vssd:VirtualSystemType></System><Item><rasd:Caption>1 virtual cpu</rasd:Caption><rasd:Description>Number of virtual CPU</rasd:Description><rasd:InstanceId>1</rasd:InstanceId><rasd:ResourceType>3</rasd:ResourceType><rasd:num_of_sockets>1</rasd:num_of_sockets><rasd:cpu_per_socket>1</rasd:cpu_per_socket></Item><Item><rasd:Caption>1024 MB of memory</rasd:Caption><rasd:Description>Memory Size</rasd:Description><rasd:InstanceId>2</rasd:InstanceId><rasd:ResourceType>4</rasd:ResourceType><rasd:AllocationUnits>MegaBytes</rasd:AllocationUnits><rasd:VirtualQuantity>1024</rasd:VirtualQuantity></Item><Item><rasd:Caption>vm1_Disk1</rasd:Caption><rasd:InstanceId>b4c44053-1991-4c45-aea6-2bcb369e4b65</rasd:InstanceId><rasd:ResourceType>17</rasd:ResourceType><rasd:HostResource>ac82bfa3-8cd5-4643-b36a-5ad28040fe0a/b4c44053-1991-4c45-aea6-2bcb369e4b65</rasd:HostResource><rasd:Parent>00000000-0000-0000-0000-000000000000</rasd:Parent><rasd:Template>00000000-0000-0000-0000-000000000000</rasd:Template><rasd:ApplicationList></rasd:ApplicationList><rasd:StorageId>572b0496-5eab-41cd-a976-4514f111898e</rasd:StorageId><rasd:StoragePoolId>aa069361-8e06-4459-a17b-748c0a2a828e</rasd:StoragePoolId><rasd:CreationDate>2014/05/13 09:48:22</rasd:CreationDate><rasd:LastModified>2014/05/13 09:48:22</rasd:LastModified><rasd:last_modified_date>2014/05/13 09:56:08</rasd:last_modified_date></Item><Item><rasd:Caption>USB Controller</rasd:Caption><rasd:InstanceId>3</rasd:InstanceId><rasd:ResourceType>23</rasd:ResourceType><rasd:UsbPolicy>DISABLED</rasd:UsbPolicy></Item></Section></Content></ovf:Envelope>', 'vm': '4da6d3ed-e2fd-4318-a58f-715cc1756430'}], sdUUID=None, options=None) >Thread-338::DEBUG::2014-05-13 12:56:09,918::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6c2e89af-083e-4583-b93a-6172c2d44419`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1345' at 'updateVM' >Thread-338::DEBUG::2014-05-13 12:56:09,919::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:56:09,919::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:56:09,919::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`6c2e89af-083e-4583-b93a-6172c2d44419`::Granted request >Thread-338::DEBUG::2014-05-13 12:56:09,919::task::827::TaskManager.Task::(resourceAcquired) Task=`4f0b806b-e21e-4979-bf22-68d3ac20cdcd`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:56:09,920::task::990::TaskManager.Task::(_decref) Task=`4f0b806b-e21e-4979-bf22-68d3ac20cdcd`::ref 1 aborting False >Thread-338::DEBUG::2014-05-13 12:56:09,920::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.4da6d3ed-e2fd-4318-a58f-715cc1756430_41830dad-f012-4b4d-b636-863f66d5ca4b`ReqID=`cdb9b5ef-dbaa-40b1-a731-8a9fb27ce644`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1353' at 'updateVM' >Thread-338::DEBUG::2014-05-13 12:56:09,920::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.4da6d3ed-e2fd-4318-a58f-715cc1756430_41830dad-f012-4b4d-b636-863f66d5ca4b' for lock type 'exclusive' >Thread-338::DEBUG::2014-05-13 12:56:09,921::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.4da6d3ed-e2fd-4318-a58f-715cc1756430_41830dad-f012-4b4d-b636-863f66d5ca4b' is free. Now locking as 'exclusive' (1 active user) >Thread-338::DEBUG::2014-05-13 12:56:09,921::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.4da6d3ed-e2fd-4318-a58f-715cc1756430_41830dad-f012-4b4d-b636-863f66d5ca4b`ReqID=`cdb9b5ef-dbaa-40b1-a731-8a9fb27ce644`::Granted request >Thread-338::DEBUG::2014-05-13 12:56:09,921::task::827::TaskManager.Task::(resourceAcquired) Task=`4f0b806b-e21e-4979-bf22-68d3ac20cdcd`::_resourcesAcquired: Storage.4da6d3ed-e2fd-4318-a58f-715cc1756430_41830dad-f012-4b4d-b636-863f66d5ca4b (exclusive) >Thread-338::DEBUG::2014-05-13 12:56:09,921::task::990::TaskManager.Task::(_decref) Task=`4f0b806b-e21e-4979-bf22-68d3ac20cdcd`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:56:09,922::sp::1237::Storage.StoragePool::(updateVM) spUUID=aa069361-8e06-4459-a17b-748c0a2a828e sdUUID=41830dad-f012-4b4d-b636-863f66d5ca4b >Thread-338::INFO::2014-05-13 12:56:09,922::sp::1248::Storage.StoragePool::(updateVM) vmUUID=4da6d3ed-e2fd-4318-a58f-715cc1756430 imgList=['ac82bfa3-8cd5-4643-b36a-5ad28040fe0a'] >Thread-338::INFO::2014-05-13 12:56:09,923::logUtils::47::dispatcher::(wrapper) Run and protect: updateVM, Return response: None >Thread-338::DEBUG::2014-05-13 12:56:09,923::task::1185::TaskManager.Task::(prepare) Task=`4f0b806b-e21e-4979-bf22-68d3ac20cdcd`::finished: None >Thread-338::DEBUG::2014-05-13 12:56:09,923::task::595::TaskManager.Task::(_updateState) Task=`4f0b806b-e21e-4979-bf22-68d3ac20cdcd`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:56:09,923::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4da6d3ed-e2fd-4318-a58f-715cc1756430_41830dad-f012-4b4d-b636-863f66d5ca4b': < ResourceRef 'Storage.4da6d3ed-e2fd-4318-a58f-715cc1756430_41830dad-f012-4b4d-b636-863f66d5ca4b', isValid: 'True' obj: 'None'>, 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:56:09,923::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:56:09,924::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.4da6d3ed-e2fd-4318-a58f-715cc1756430_41830dad-f012-4b4d-b636-863f66d5ca4b' >Thread-338::DEBUG::2014-05-13 12:56:09,924::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.4da6d3ed-e2fd-4318-a58f-715cc1756430_41830dad-f012-4b4d-b636-863f66d5ca4b' (0 active users) >Thread-338::DEBUG::2014-05-13 12:56:09,924::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.4da6d3ed-e2fd-4318-a58f-715cc1756430_41830dad-f012-4b4d-b636-863f66d5ca4b' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:56:09,924::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4da6d3ed-e2fd-4318-a58f-715cc1756430_41830dad-f012-4b4d-b636-863f66d5ca4b', Clearing records. >Thread-338::DEBUG::2014-05-13 12:56:09,924::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:56:09,925::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:56:09,925::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:56:09,925::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:56:09,925::task::990::TaskManager.Task::(_decref) Task=`4f0b806b-e21e-4979-bf22-68d3ac20cdcd`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:56:10,031::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:56:10,031::task::595::TaskManager.Task::(_updateState) Task=`97762570-fbe3-4ee6-bee1-2f9461aef91f`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:56:10,032::logUtils::44::dispatcher::(wrapper) Run and protect: updateVM(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', vmList=[{'imglist': 'f35efc5c-cd67-4de9-b1c6-9e08bafcae46', 'ovf': '<?xml version=\'1.0\' encoding=\'UTF-8\'?><ovf:Envelope xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1/" xmlns:rasd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData" xmlns:vssd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" ovf:version="3.4.0.0"><References><File ovf:href="f35efc5c-cd67-4de9-b1c6-9e08bafcae46/c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a" ovf:id="c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a" ovf:size="5368709120" ovf:description=""/></References><Section xsi:type="ovf:NetworkSection_Type"><Info>List of networks</Info><Network ovf:name="Network 1"/></Section><Section xsi:type="ovf:DiskSection_Type"><Info>List of Virtual Disks</Info><Disk ovf:diskId="c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a" ovf:size="5" ovf:actual_size="0" ovf:vm_snapshot_id="b222d608-69f0-4204-9731-06400ba31083" ovf:parentRef="" ovf:fileRef="f35efc5c-cd67-4de9-b1c6-9e08bafcae46/c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a" ovf:format="http://www.vmware.com/specifications/vmdk.html#sparse" ovf:volume-format="RAW" ovf:volume-type="Sparse" ovf:disk-interface="VirtIO_SCSI" ovf:boot="true" ovf:disk-alias="vm1_Disk1" ovf:wipe-after-delete="false"/></Section><Content ovf:id="out" xsi:type="ovf:VirtualSystem_Type"><Description></Description><CreationDate>2014/05/13 09:49:27</CreationDate><ExportDate>2014/05/13 09:56:08</ExportDate><DeleteProtected>false</DeleteProtected><SsoMethod>guest_agent</SsoMethod><IsSmartcardEnabled>false</IsSmartcardEnabled><TimeZone></TimeZone><default_boot_sequence>0</default_boot_sequence><Generation>2</Generation><VmType>1</VmType><MinAllocatedMem>1024</MinAllocatedMem><IsStateless>false</IsStateless><IsRunAndPause>false</IsRunAndPause><AutoStartup>false</AutoStartup><Priority>1</Priority><CreatedByUserId>fdfc627c-d875-11e0-90f0-83df133b58cc</CreatedByUserId><Name>temp</Name><TemplateId>e953f9c0-078e-4548-bcf2-0d1e796896e2</TemplateId><Origin>3</Origin><default_display_type>1</default_display_type><IsDisabled>false</IsDisabled><TrustedService>false</TrustedService><TemplateType>TEMPLATE</TemplateType><BaseTemplateId>e953f9c0-078e-4548-bcf2-0d1e796896e2</BaseTemplateId><TemplateVersionNumber>1</TemplateVersionNumber><TemplateVersionName>base version</TemplateVersionName><Section ovf:id="e953f9c0-078e-4548-bcf2-0d1e796896e2" ovf:required="false" xsi:type="ovf:OperatingSystemSection_Type"><Info>Guest Operating System</Info><Description>other</Description></Section><Section xsi:type="ovf:VirtualHardwareSection_Type"><Info>1 CPU, 1024 Memeory</Info><System><vssd:VirtualSystemType>ENGINE 3.4.0.0</vssd:VirtualSystemType></System><Item><rasd:Caption>1 virtual cpu</rasd:Caption><rasd:Description>Number of virtual CPU</rasd:Description><rasd:InstanceId>1</rasd:InstanceId><rasd:ResourceType>3</rasd:ResourceType><rasd:num_of_sockets>1</rasd:num_of_sockets><rasd:cpu_per_socket>1</rasd:cpu_per_socket></Item><Item><rasd:Caption>1024 MB of memory</rasd:Caption><rasd:Description>Memory Size</rasd:Description><rasd:InstanceId>2</rasd:InstanceId><rasd:ResourceType>4</rasd:ResourceType><rasd:AllocationUnits>MegaBytes</rasd:AllocationUnits><rasd:VirtualQuantity>1024</rasd:VirtualQuantity></Item><Item><rasd:Caption>vm1_Disk1</rasd:Caption><rasd:InstanceId>c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a</rasd:InstanceId><rasd:ResourceType>17</rasd:ResourceType><rasd:HostResource>f35efc5c-cd67-4de9-b1c6-9e08bafcae46/c7bcb02a-80b4-48e3-8df9-c1a2cbeb716a</rasd:HostResource><rasd:Parent>00000000-0000-0000-0000-000000000000</rasd:Parent><rasd:Template>00000000-0000-0000-0000-000000000000</rasd:Template><rasd:ApplicationList></rasd:ApplicationList><rasd:StorageId>572b0496-5eab-41cd-a976-4514f111898e</rasd:StorageId><rasd:StoragePoolId>aa069361-8e06-4459-a17b-748c0a2a828e</rasd:StoragePoolId><rasd:CreationDate>2014/05/13 09:49:29</rasd:CreationDate><rasd:LastModified>2014/05/13 09:49:29</rasd:LastModified></Item><Item><rasd:Caption>USB Controller</rasd:Caption><rasd:InstanceId>3</rasd:InstanceId><rasd:ResourceType>23</rasd:ResourceType><rasd:UsbPolicy>DISABLED</rasd:UsbPolicy></Item></Section></Content></ovf:Envelope>', 'vm': 'e953f9c0-078e-4548-bcf2-0d1e796896e2'}], sdUUID=None, options=None) >Thread-338::DEBUG::2014-05-13 12:56:10,032::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b4dafd62-002a-4305-8324-566dd9cdffd6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1345' at 'updateVM' >Thread-338::DEBUG::2014-05-13 12:56:10,032::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:56:10,032::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:56:10,033::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b4dafd62-002a-4305-8324-566dd9cdffd6`::Granted request >Thread-338::DEBUG::2014-05-13 12:56:10,033::task::827::TaskManager.Task::(resourceAcquired) Task=`97762570-fbe3-4ee6-bee1-2f9461aef91f`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:56:10,033::task::990::TaskManager.Task::(_decref) Task=`97762570-fbe3-4ee6-bee1-2f9461aef91f`::ref 1 aborting False >Thread-338::DEBUG::2014-05-13 12:56:10,034::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.e953f9c0-078e-4548-bcf2-0d1e796896e2_41830dad-f012-4b4d-b636-863f66d5ca4b`ReqID=`091b2cd2-3829-45a6-b297-27797bbf0631`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1353' at 'updateVM' >Thread-338::DEBUG::2014-05-13 12:56:10,034::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.e953f9c0-078e-4548-bcf2-0d1e796896e2_41830dad-f012-4b4d-b636-863f66d5ca4b' for lock type 'exclusive' >Thread-338::DEBUG::2014-05-13 12:56:10,034::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.e953f9c0-078e-4548-bcf2-0d1e796896e2_41830dad-f012-4b4d-b636-863f66d5ca4b' is free. Now locking as 'exclusive' (1 active user) >Thread-338::DEBUG::2014-05-13 12:56:10,034::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.e953f9c0-078e-4548-bcf2-0d1e796896e2_41830dad-f012-4b4d-b636-863f66d5ca4b`ReqID=`091b2cd2-3829-45a6-b297-27797bbf0631`::Granted request >Thread-338::DEBUG::2014-05-13 12:56:10,035::task::827::TaskManager.Task::(resourceAcquired) Task=`97762570-fbe3-4ee6-bee1-2f9461aef91f`::_resourcesAcquired: Storage.e953f9c0-078e-4548-bcf2-0d1e796896e2_41830dad-f012-4b4d-b636-863f66d5ca4b (exclusive) >Thread-338::DEBUG::2014-05-13 12:56:10,035::task::990::TaskManager.Task::(_decref) Task=`97762570-fbe3-4ee6-bee1-2f9461aef91f`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:56:10,035::sp::1237::Storage.StoragePool::(updateVM) spUUID=aa069361-8e06-4459-a17b-748c0a2a828e sdUUID=41830dad-f012-4b4d-b636-863f66d5ca4b >Thread-338::INFO::2014-05-13 12:56:10,035::sp::1248::Storage.StoragePool::(updateVM) vmUUID=e953f9c0-078e-4548-bcf2-0d1e796896e2 imgList=['f35efc5c-cd67-4de9-b1c6-9e08bafcae46'] >Thread-338::INFO::2014-05-13 12:56:10,036::logUtils::47::dispatcher::(wrapper) Run and protect: updateVM, Return response: None >Thread-338::DEBUG::2014-05-13 12:56:10,036::task::1185::TaskManager.Task::(prepare) Task=`97762570-fbe3-4ee6-bee1-2f9461aef91f`::finished: None >Thread-338::DEBUG::2014-05-13 12:56:10,036::task::595::TaskManager.Task::(_updateState) Task=`97762570-fbe3-4ee6-bee1-2f9461aef91f`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:56:10,036::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.e953f9c0-078e-4548-bcf2-0d1e796896e2_41830dad-f012-4b4d-b636-863f66d5ca4b': < ResourceRef 'Storage.e953f9c0-078e-4548-bcf2-0d1e796896e2_41830dad-f012-4b4d-b636-863f66d5ca4b', isValid: 'True' obj: 'None'>, 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:56:10,036::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:56:10,037::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.e953f9c0-078e-4548-bcf2-0d1e796896e2_41830dad-f012-4b4d-b636-863f66d5ca4b' >Thread-338::DEBUG::2014-05-13 12:56:10,037::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.e953f9c0-078e-4548-bcf2-0d1e796896e2_41830dad-f012-4b4d-b636-863f66d5ca4b' (0 active users) >Thread-338::DEBUG::2014-05-13 12:56:10,037::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.e953f9c0-078e-4548-bcf2-0d1e796896e2_41830dad-f012-4b4d-b636-863f66d5ca4b' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:56:10,037::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.e953f9c0-078e-4548-bcf2-0d1e796896e2_41830dad-f012-4b4d-b636-863f66d5ca4b', Clearing records. >Thread-338::DEBUG::2014-05-13 12:56:10,037::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:56:10,038::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:56:10,038::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:56:10,038::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:56:10,038::task::990::TaskManager.Task::(_decref) Task=`97762570-fbe3-4ee6-bee1-2f9461aef91f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:56:10,050::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:56:10,050::task::595::TaskManager.Task::(_updateState) Task=`b75e6466-07e5-4fec-832f-0ddcb31a87fc`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:56:10,050::logUtils::44::dispatcher::(wrapper) Run and protect: removeVM(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', vmUUID='ada2f983-302f-4302-9243-f1254e379928', sdUUID=None, options=None) >Thread-338::DEBUG::2014-05-13 12:56:10,050::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f1bf56a1-2beb-4066-bbcb-77022deaa5d3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1371' at 'removeVM' >Thread-338::DEBUG::2014-05-13 12:56:10,051::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:56:10,051::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:56:10,051::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`f1bf56a1-2beb-4066-bbcb-77022deaa5d3`::Granted request >Thread-338::DEBUG::2014-05-13 12:56:10,051::task::827::TaskManager.Task::(resourceAcquired) Task=`b75e6466-07e5-4fec-832f-0ddcb31a87fc`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:56:10,052::task::990::TaskManager.Task::(_decref) Task=`b75e6466-07e5-4fec-832f-0ddcb31a87fc`::ref 1 aborting False >Thread-338::DEBUG::2014-05-13 12:56:10,052::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.ada2f983-302f-4302-9243-f1254e379928_41830dad-f012-4b4d-b636-863f66d5ca4b`ReqID=`f8c388c7-d69c-4bed-8b3d-4d99a2ba7aa8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1375' at 'removeVM' >Thread-338::DEBUG::2014-05-13 12:56:10,052::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.ada2f983-302f-4302-9243-f1254e379928_41830dad-f012-4b4d-b636-863f66d5ca4b' for lock type 'exclusive' >Thread-338::DEBUG::2014-05-13 12:56:10,052::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.ada2f983-302f-4302-9243-f1254e379928_41830dad-f012-4b4d-b636-863f66d5ca4b' is free. Now locking as 'exclusive' (1 active user) >Thread-338::DEBUG::2014-05-13 12:56:10,053::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.ada2f983-302f-4302-9243-f1254e379928_41830dad-f012-4b4d-b636-863f66d5ca4b`ReqID=`f8c388c7-d69c-4bed-8b3d-4d99a2ba7aa8`::Granted request >Thread-338::DEBUG::2014-05-13 12:56:10,053::task::827::TaskManager.Task::(resourceAcquired) Task=`b75e6466-07e5-4fec-832f-0ddcb31a87fc`::_resourcesAcquired: Storage.ada2f983-302f-4302-9243-f1254e379928_41830dad-f012-4b4d-b636-863f66d5ca4b (exclusive) >Thread-338::DEBUG::2014-05-13 12:56:10,053::task::990::TaskManager.Task::(_decref) Task=`b75e6466-07e5-4fec-832f-0ddcb31a87fc`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:56:10,053::sp::1276::Storage.StoragePool::(removeVM) spUUID=aa069361-8e06-4459-a17b-748c0a2a828e vmUUID=ada2f983-302f-4302-9243-f1254e379928 sdUUID=41830dad-f012-4b4d-b636-863f66d5ca4b >Thread-338::INFO::2014-05-13 12:56:10,054::logUtils::47::dispatcher::(wrapper) Run and protect: removeVM, Return response: None >Thread-338::DEBUG::2014-05-13 12:56:10,054::task::1185::TaskManager.Task::(prepare) Task=`b75e6466-07e5-4fec-832f-0ddcb31a87fc`::finished: None >Thread-338::DEBUG::2014-05-13 12:56:10,054::task::595::TaskManager.Task::(_updateState) Task=`b75e6466-07e5-4fec-832f-0ddcb31a87fc`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:56:10,054::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.ada2f983-302f-4302-9243-f1254e379928_41830dad-f012-4b4d-b636-863f66d5ca4b': < ResourceRef 'Storage.ada2f983-302f-4302-9243-f1254e379928_41830dad-f012-4b4d-b636-863f66d5ca4b', isValid: 'True' obj: 'None'>, 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:56:10,054::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:56:10,055::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.ada2f983-302f-4302-9243-f1254e379928_41830dad-f012-4b4d-b636-863f66d5ca4b' >Thread-338::DEBUG::2014-05-13 12:56:10,055::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.ada2f983-302f-4302-9243-f1254e379928_41830dad-f012-4b4d-b636-863f66d5ca4b' (0 active users) >Thread-338::DEBUG::2014-05-13 12:56:10,055::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.ada2f983-302f-4302-9243-f1254e379928_41830dad-f012-4b4d-b636-863f66d5ca4b' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:56:10,055::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ada2f983-302f-4302-9243-f1254e379928_41830dad-f012-4b4d-b636-863f66d5ca4b', Clearing records. >Thread-338::DEBUG::2014-05-13 12:56:10,056::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:56:10,056::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:56:10,056::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:56:10,056::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:56:10,056::task::990::TaskManager.Task::(_decref) Task=`b75e6466-07e5-4fec-832f-0ddcb31a87fc`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:56:11,419::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:56:11,498::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353207 s, 29.0 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:56:11,918::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 208946c2-5db1-4ef4-80b1-9aeb37f2905e >Thread-357::DEBUG::2014-05-13 12:56:11,928::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e >Thread-357::DEBUG::2014-05-13 12:56:11,929::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-357::DEBUG::2014-05-13 12:56:11,937::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=ISO_DOMAIN', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=aa069361-8e06-4459-a17b-748c0a2a828e', 'REMOTE_PATH=dhcp163-66.scl.lab.tlv.redhat.com:/var/lib/exports/iso-20140512085215', 'ROLE=Regular', 'SDUUID=208946c2-5db1-4ef4-80b1-9aeb37f2905e', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=0be9ca2275e0b4f627ac4aeccb1d1475df0d42f5'] >Thread-357::DEBUG::2014-05-13 12:56:11,939::fileSD::609::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-357::INFO::2014-05-13 12:56:11,939::sd::384::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 208946c2-5db1-4ef4-80b1-9aeb37f2905e_imageNS already registered >Thread-357::INFO::2014-05-13 12:56:11,939::sd::392::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 208946c2-5db1-4ef4-80b1-9aeb37f2905e_volumeNS already registered >Thread-357::DEBUG::2014-05-13 12:56:11,949::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:56:11,958::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000564918 s, 692 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:56:12,385::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:56:13,511::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:56:13,586::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310033 s, 33.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:56:13,640::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:56:13,641::task::595::TaskManager.Task::(_updateState) Task=`37854ad4-ea41-4640-bfcb-3d14f1287d94`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:56:13,641::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:56:13,641::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:56:13,641::task::1185::TaskManager.Task::(prepare) Task=`37854ad4-ea41-4640-bfcb-3d14f1287d94`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:56:13,642::task::595::TaskManager.Task::(_updateState) Task=`37854ad4-ea41-4640-bfcb-3d14f1287d94`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:56:13,642::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:56:13,642::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:56:13,642::task::990::TaskManager.Task::(_decref) Task=`37854ad4-ea41-4640-bfcb-3d14f1287d94`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:56:13,648::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:56:13,648::task::595::TaskManager.Task::(_updateState) Task=`4d99e945-e35b-44d2-a187-33154320b419`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:56:13,648::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:56:13,649::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b34b6e9e-8cc6-468a-99e9-59ac1b5d1ee1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:56:13,649::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:56:13,649::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:56:13,649::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`b34b6e9e-8cc6-468a-99e9-59ac1b5d1ee1`::Granted request >Thread-338::DEBUG::2014-05-13 12:56:13,650::task::827::TaskManager.Task::(resourceAcquired) Task=`4d99e945-e35b-44d2-a187-33154320b419`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:56:13,650::task::990::TaskManager.Task::(_decref) Task=`4d99e945-e35b-44d2-a187-33154320b419`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:56:13,651::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:56:13,651::task::1185::TaskManager.Task::(prepare) Task=`4d99e945-e35b-44d2-a187-33154320b419`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:56:13,651::task::595::TaskManager.Task::(_updateState) Task=`4d99e945-e35b-44d2-a187-33154320b419`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:56:13,651::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:56:13,652::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:56:13,652::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:56:13,652::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:56:13,652::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:56:13,653::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:56:13,653::task::990::TaskManager.Task::(_decref) Task=`4d99e945-e35b-44d2-a187-33154320b419`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:56:13,920::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:56:13,930::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000550839 s, 632 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:56:15,404::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:56:15,599::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:56:15,683::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0390641 s, 26.2 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:56:17,696::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:56:17,771::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0329236 s, 31.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:56:18,422::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:56:19,215::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:56:19,223::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000560351 s, 7.3 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:56:19,784::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-66::DEBUG::2014-05-13 12:56:19,838::task::595::TaskManager.Task::(_updateState) Task=`f6f47ab5-3721-458f-aadc-8cb2616ed957`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:56:19,838::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:56:19,838::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000564918', 'lastCheck': '7.9', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000560351', 'lastCheck': '0.6', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000550839', 'lastCheck': '5.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:56:19,838::task::1185::TaskManager.Task::(prepare) Task=`f6f47ab5-3721-458f-aadc-8cb2616ed957`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000564918', 'lastCheck': '7.9', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000560351', 'lastCheck': '0.6', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000550839', 'lastCheck': '5.9', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:56:19,839::task::595::TaskManager.Task::(_updateState) Task=`f6f47ab5-3721-458f-aadc-8cb2616ed957`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:56:19,839::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:56:19,839::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:56:19,839::task::990::TaskManager.Task::(_decref) Task=`f6f47ab5-3721-458f-aadc-8cb2616ed957`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:56:20,142::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.315796 s, 3.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:56:21,439::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-357::DEBUG::2014-05-13 12:56:21,969::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:56:21,979::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000769704 s, 508 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:56:22,155::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:56:22,235::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361272 s, 28.3 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:56:23,697::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [28fd42b9] >Thread-66::DEBUG::2014-05-13 12:56:23,697::task::595::TaskManager.Task::(_updateState) Task=`809b84e1-2089-4c9e-a99d-fe3aced09fdc`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:56:23,697::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:56:23,698::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:56:23,698::task::1185::TaskManager.Task::(prepare) Task=`809b84e1-2089-4c9e-a99d-fe3aced09fdc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:56:23,698::task::595::TaskManager.Task::(_updateState) Task=`809b84e1-2089-4c9e-a99d-fe3aced09fdc`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:56:23,698::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:56:23,698::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:56:23,698::task::990::TaskManager.Task::(_decref) Task=`809b84e1-2089-4c9e-a99d-fe3aced09fdc`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:56:23,703::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [28fd42b9] >Thread-338::DEBUG::2014-05-13 12:56:23,703::task::595::TaskManager.Task::(_updateState) Task=`776dbf1d-bea9-4e95-96a8-8a0960b3018e`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:56:23,703::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:56:23,704::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`82cddbb0-5ec2-4e24-ae86-1d7e65a029fb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:56:23,704::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:56:23,704::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:56:23,704::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`82cddbb0-5ec2-4e24-ae86-1d7e65a029fb`::Granted request >Thread-338::DEBUG::2014-05-13 12:56:23,705::task::827::TaskManager.Task::(resourceAcquired) Task=`776dbf1d-bea9-4e95-96a8-8a0960b3018e`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:56:23,705::task::990::TaskManager.Task::(_decref) Task=`776dbf1d-bea9-4e95-96a8-8a0960b3018e`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:56:23,706::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:56:23,706::task::1185::TaskManager.Task::(prepare) Task=`776dbf1d-bea9-4e95-96a8-8a0960b3018e`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:56:23,706::task::595::TaskManager.Task::(_updateState) Task=`776dbf1d-bea9-4e95-96a8-8a0960b3018e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:56:23,706::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:56:23,706::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:56:23,707::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:56:23,707::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:56:23,707::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:56:23,707::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:56:23,707::task::990::TaskManager.Task::(_decref) Task=`776dbf1d-bea9-4e95-96a8-8a0960b3018e`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:56:23,939::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:56:23,948::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000492504 s, 707 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:56:24,249::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:56:24,328::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0360647 s, 28.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:56:24,457::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:56:26,341::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:56:26,420::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353775 s, 28.9 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:56:27,483::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:56:28,433::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:56:28,510::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0313542 s, 32.7 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:56:29,231::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:56:29,241::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00162076 s, 2.5 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:56:30,502::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:56:30,524::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:56:30,700::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.12438 s, 8.2 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:56:31,989::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:56:31,998::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000648673 s, 603 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:56:32,715::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:56:32,807::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0478274 s, 21.4 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:56:33,519::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:56:33,753::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:56:33,753::task::595::TaskManager.Task::(_updateState) Task=`58cdd029-b17b-434d-af55-ae89bce06113`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:56:33,754::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:56:33,754::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:56:33,754::task::1185::TaskManager.Task::(prepare) Task=`58cdd029-b17b-434d-af55-ae89bce06113`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:56:33,754::task::595::TaskManager.Task::(_updateState) Task=`58cdd029-b17b-434d-af55-ae89bce06113`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:56:33,755::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:56:33,755::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:56:33,755::task::990::TaskManager.Task::(_decref) Task=`58cdd029-b17b-434d-af55-ae89bce06113`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:56:33,760::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:56:33,760::task::595::TaskManager.Task::(_updateState) Task=`11a10fc9-777b-432c-a78a-b9a3844ad070`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:56:33,760::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:56:33,761::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4bbe44ce-2510-4b50-bfe1-77fd09362e78`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:56:33,761::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:56:33,761::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:56:33,761::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`4bbe44ce-2510-4b50-bfe1-77fd09362e78`::Granted request >Thread-338::DEBUG::2014-05-13 12:56:33,762::task::827::TaskManager.Task::(resourceAcquired) Task=`11a10fc9-777b-432c-a78a-b9a3844ad070`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:56:33,762::task::990::TaskManager.Task::(_decref) Task=`11a10fc9-777b-432c-a78a-b9a3844ad070`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:56:33,763::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:56:33,763::task::1185::TaskManager.Task::(prepare) Task=`11a10fc9-777b-432c-a78a-b9a3844ad070`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:56:33,763::task::595::TaskManager.Task::(_updateState) Task=`11a10fc9-777b-432c-a78a-b9a3844ad070`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:56:33,764::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:56:33,764::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:56:33,764::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:56:33,764::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:56:33,764::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:56:33,765::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:56:33,765::task::990::TaskManager.Task::(_decref) Task=`11a10fc9-777b-432c-a78a-b9a3844ad070`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:56:33,959::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:56:33,968::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000402653 s, 864 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:56:34,820::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:56:34,896::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312352 s, 32.8 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:56:34,991::task::595::TaskManager.Task::(_updateState) Task=`4e7ec019-652e-45cd-b632-67dd637398d1`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:56:34,991::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:56:34,992::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000648673', 'lastCheck': '3.0', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00162076', 'lastCheck': '5.7', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000402653', 'lastCheck': '1.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:56:34,992::task::1185::TaskManager.Task::(prepare) Task=`4e7ec019-652e-45cd-b632-67dd637398d1`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000648673', 'lastCheck': '3.0', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00162076', 'lastCheck': '5.7', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000402653', 'lastCheck': '1.0', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:56:34,992::task::595::TaskManager.Task::(_updateState) Task=`4e7ec019-652e-45cd-b632-67dd637398d1`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:56:34,992::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:56:34,992::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:56:34,993::task::990::TaskManager.Task::(_decref) Task=`4e7ec019-652e-45cd-b632-67dd637398d1`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:56:36,536::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:56:36,908::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:56:36,989::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0354735 s, 28.9 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:56:39,003::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:56:39,096::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0308434 s, 33.2 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:56:39,249::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:56:39,258::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000674126 s, 6.1 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:56:39,556::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:56:41,109::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:56:41,189::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037829 s, 27.1 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:56:42,010::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:56:42,019::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000479924 s, 815 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:56:42,581::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:56:43,202::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:56:43,282::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037378 s, 27.4 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:56:43,804::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:56:43,804::task::595::TaskManager.Task::(_updateState) Task=`6396ed8b-0c5e-4263-960e-298ac58bf7d1`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:56:43,804::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:56:43,805::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:56:43,805::task::1185::TaskManager.Task::(prepare) Task=`6396ed8b-0c5e-4263-960e-298ac58bf7d1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:56:43,805::task::595::TaskManager.Task::(_updateState) Task=`6396ed8b-0c5e-4263-960e-298ac58bf7d1`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:56:43,805::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:56:43,805::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:56:43,805::task::990::TaskManager.Task::(_decref) Task=`6396ed8b-0c5e-4263-960e-298ac58bf7d1`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:56:43,810::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:56:43,810::task::595::TaskManager.Task::(_updateState) Task=`039064b7-d4b5-42c6-b9d4-e1562a1d7594`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:56:43,810::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:56:43,812::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`645e1b77-56d5-4655-aeba-0aa3b137364d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:56:43,812::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:56:43,813::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:56:43,813::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`645e1b77-56d5-4655-aeba-0aa3b137364d`::Granted request >Thread-338::DEBUG::2014-05-13 12:56:43,813::task::827::TaskManager.Task::(resourceAcquired) Task=`039064b7-d4b5-42c6-b9d4-e1562a1d7594`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:56:43,813::task::990::TaskManager.Task::(_decref) Task=`039064b7-d4b5-42c6-b9d4-e1562a1d7594`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:56:43,814::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:56:43,814::task::1185::TaskManager.Task::(prepare) Task=`039064b7-d4b5-42c6-b9d4-e1562a1d7594`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:56:43,814::task::595::TaskManager.Task::(_updateState) Task=`039064b7-d4b5-42c6-b9d4-e1562a1d7594`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:56:43,815::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:56:43,815::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:56:43,815::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:56:43,815::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:56:43,815::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:56:43,815::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:56:43,816::task::990::TaskManager.Task::(_decref) Task=`039064b7-d4b5-42c6-b9d4-e1562a1d7594`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:56:43,978::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:56:43,987::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000498703 s, 698 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:56:45,290::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >BindingXMLRPC::ERROR::2014-05-13 12:56:45,601::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:56:45,703::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.370716 s, 2.8 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:56:47,716::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:56:47,790::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0304368 s, 33.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:56:48,624::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-322::DEBUG::2014-05-13 12:56:49,267::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:56:49,276::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000588607 s, 7.0 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:56:49,803::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:56:49,878::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319728 s, 32.0 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:56:50,129::task::595::TaskManager.Task::(_updateState) Task=`ec91ef70-41df-477d-ac9f-d4df2146d007`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:56:50,129::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:56:50,130::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000479924', 'lastCheck': '8.1', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000588607', 'lastCheck': '0.8', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000498703', 'lastCheck': '6.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:56:50,130::task::1185::TaskManager.Task::(prepare) Task=`ec91ef70-41df-477d-ac9f-d4df2146d007`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000479924', 'lastCheck': '8.1', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000588607', 'lastCheck': '0.8', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000498703', 'lastCheck': '6.1', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:56:50,130::task::595::TaskManager.Task::(_updateState) Task=`ec91ef70-41df-477d-ac9f-d4df2146d007`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:56:50,130::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:56:50,130::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:56:50,131::task::990::TaskManager.Task::(_decref) Task=`ec91ef70-41df-477d-ac9f-d4df2146d007`::ref 0 aborting False >BindingXMLRPC::ERROR::2014-05-13 12:56:51,641::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:56:51,891::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:56:51,974::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0373618 s, 27.4 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:56:52,031::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:56:52,040::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000448517 s, 872 kB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:56:53,850::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6602059] >Thread-66::DEBUG::2014-05-13 12:56:53,850::task::595::TaskManager.Task::(_updateState) Task=`01a77239-b6b3-46c5-a8d3-54b83244744d`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:56:53,850::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:56:53,850::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:56:53,851::task::1185::TaskManager.Task::(prepare) Task=`01a77239-b6b3-46c5-a8d3-54b83244744d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:56:53,851::task::595::TaskManager.Task::(_updateState) Task=`01a77239-b6b3-46c5-a8d3-54b83244744d`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:56:53,851::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:56:53,851::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:56:53,851::task::990::TaskManager.Task::(_decref) Task=`01a77239-b6b3-46c5-a8d3-54b83244744d`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:56:53,856::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [6602059] >Thread-338::DEBUG::2014-05-13 12:56:53,856::task::595::TaskManager.Task::(_updateState) Task=`d3f621db-02ca-44cc-a7eb-aae6c8dfc907`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:56:53,856::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:56:53,856::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9be683dc-0751-4401-9be2-84643018725d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:56:53,857::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:56:53,857::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:56:53,857::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`9be683dc-0751-4401-9be2-84643018725d`::Granted request >Thread-338::DEBUG::2014-05-13 12:56:53,857::task::827::TaskManager.Task::(resourceAcquired) Task=`d3f621db-02ca-44cc-a7eb-aae6c8dfc907`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:56:53,857::task::990::TaskManager.Task::(_decref) Task=`d3f621db-02ca-44cc-a7eb-aae6c8dfc907`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:56:53,858::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:56:53,858::task::1185::TaskManager.Task::(prepare) Task=`d3f621db-02ca-44cc-a7eb-aae6c8dfc907`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:56:53,859::task::595::TaskManager.Task::(_updateState) Task=`d3f621db-02ca-44cc-a7eb-aae6c8dfc907`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:56:53,859::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:56:53,859::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:56:53,859::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:56:53,859::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:56:53,859::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:56:53,860::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:56:53,860::task::990::TaskManager.Task::(_decref) Task=`d3f621db-02ca-44cc-a7eb-aae6c8dfc907`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:56:53,986::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Thread-344::DEBUG::2014-05-13 12:56:54,008::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:56:54,026::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000374241 s, 930 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:56:54,092::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0641104 s, 16.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:56:54,675::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:56:56,105::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:56:56,187::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374631 s, 27.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:56:57,698::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:56:58,200::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:56:58,295::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0498884 s, 20.5 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:56:59,284::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:56:59,292::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000551918 s, 7.4 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:57:00,304::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:57:00,383::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356264 s, 28.7 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:57:00,716::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-357::DEBUG::2014-05-13 12:57:02,052::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:57:02,062::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.00036826 s, 1.1 MB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:57:02,396::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:57:02,476::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0376494 s, 27.2 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:57:03,735::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Thread-66::DEBUG::2014-05-13 12:57:03,898::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [28fd42b9] >Thread-66::DEBUG::2014-05-13 12:57:03,899::task::595::TaskManager.Task::(_updateState) Task=`b2f4f045-ca4e-42b0-a35a-d0ca0f57e1d3`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:57:03,899::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:57:03,899::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:57:03,900::task::1185::TaskManager.Task::(prepare) Task=`b2f4f045-ca4e-42b0-a35a-d0ca0f57e1d3`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:57:03,900::task::595::TaskManager.Task::(_updateState) Task=`b2f4f045-ca4e-42b0-a35a-d0ca0f57e1d3`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:57:03,900::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:57:03,900::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:57:03,900::task::990::TaskManager.Task::(_decref) Task=`b2f4f045-ca4e-42b0-a35a-d0ca0f57e1d3`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:57:03,906::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] flowID [28fd42b9] >Thread-338::DEBUG::2014-05-13 12:57:03,906::task::595::TaskManager.Task::(_updateState) Task=`8fe111ce-4348-40d9-a51c-ca3bb0065d61`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:57:03,906::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:57:03,907::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ca956f2e-26ba-41d3-a904-fee8d4b82360`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:57:03,907::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:57:03,907::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:57:03,907::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`ca956f2e-26ba-41d3-a904-fee8d4b82360`::Granted request >Thread-338::DEBUG::2014-05-13 12:57:03,908::task::827::TaskManager.Task::(resourceAcquired) Task=`8fe111ce-4348-40d9-a51c-ca3bb0065d61`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:57:03,908::task::990::TaskManager.Task::(_decref) Task=`8fe111ce-4348-40d9-a51c-ca3bb0065d61`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:57:03,909::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:57:03,909::task::1185::TaskManager.Task::(prepare) Task=`8fe111ce-4348-40d9-a51c-ca3bb0065d61`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:57:03,909::task::595::TaskManager.Task::(_updateState) Task=`8fe111ce-4348-40d9-a51c-ca3bb0065d61`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:57:03,910::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:57:03,910::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:57:03,910::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:57:03,910::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:57:03,910::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:57:03,910::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:57:03,911::task::990::TaskManager.Task::(_decref) Task=`8fe111ce-4348-40d9-a51c-ca3bb0065d61`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:57:04,041::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:57:04,050::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000417437 s, 834 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:57:04,489::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:57:04,564::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0329038 s, 31.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:57:05,266::task::595::TaskManager.Task::(_updateState) Task=`940ede95-2bf9-4936-97dc-fbe1ebdf68f3`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:57:05,267::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-05-13 12:57:05,267::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00036826', 'lastCheck': '3.2', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000551918', 'lastCheck': '6.0', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417437', 'lastCheck': '1.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:57:05,267::task::1185::TaskManager.Task::(prepare) Task=`940ede95-2bf9-4936-97dc-fbe1ebdf68f3`::finished: {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00036826', 'lastCheck': '3.2', 'valid': True}, '41830dad-f012-4b4d-b636-863f66d5ca4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000551918', 'lastCheck': '6.0', 'valid': True}, '572b0496-5eab-41cd-a976-4514f111898e': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417437', 'lastCheck': '1.2', 'valid': True}} >Thread-66::DEBUG::2014-05-13 12:57:05,267::task::595::TaskManager.Task::(_updateState) Task=`940ede95-2bf9-4936-97dc-fbe1ebdf68f3`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:57:05,268::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:57:05,268::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:57:05,268::task::990::TaskManager.Task::(_decref) Task=`940ede95-2bf9-4936-97dc-fbe1ebdf68f3`::ref 0 aborting False >Dummy-337::DEBUG::2014-05-13 12:57:06,577::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:57:06,661::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0393105 s, 26.0 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:57:06,752::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:57:08,674::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:57:08,750::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031583 s, 32.4 MB/s\n'; <rc> = 0 >Thread-322::DEBUG::2014-05-13 12:57:09,307::blockSD::600::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/dev/41830dad-f012-4b4d-b636-863f66d5ca4b/metadata bs=4096 count=1' (cwd None) >Thread-322::DEBUG::2014-05-13 12:57:09,316::blockSD::600::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00076562 s, 5.3 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:57:09,786::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:57:10,763::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:57:10,845::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0396233 s, 25.8 MB/s\n'; <rc> = 0 >Thread-357::DEBUG::2014-05-13 12:57:12,075::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-357::DEBUG::2014-05-13 12:57:12,084::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n391 bytes (391 B) copied, 0.000496883 s, 787 kB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:57:12,803::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:57:12,859::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:57:13,241::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.332665 s, 3.1 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-05-13 12:57:13,949::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-66::DEBUG::2014-05-13 12:57:13,996::task::595::TaskManager.Task::(_updateState) Task=`631ad8c2-1db5-4d9f-a8bf-ae0d97dcad2f`::moving from state init -> state preparing >Thread-66::INFO::2014-05-13 12:57:13,996::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-66::INFO::2014-05-13 12:57:13,996::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:57:13,997::task::1185::TaskManager.Task::(prepare) Task=`631ad8c2-1db5-4d9f-a8bf-ae0d97dcad2f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-66::DEBUG::2014-05-13 12:57:13,997::task::595::TaskManager.Task::(_updateState) Task=`631ad8c2-1db5-4d9f-a8bf-ae0d97dcad2f`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-05-13 12:57:13,997::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-05-13 12:57:13,997::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-05-13 12:57:13,997::task::990::TaskManager.Task::(_decref) Task=`631ad8c2-1db5-4d9f-a8bf-ae0d97dcad2f`::ref 0 aborting False >Thread-338::DEBUG::2014-05-13 12:57:14,002::BindingXMLRPC::251::vds::(wrapper) client [10.35.163.66] >Thread-338::DEBUG::2014-05-13 12:57:14,002::task::595::TaskManager.Task::(_updateState) Task=`6014a953-12af-4c7a-b595-b20be0acecae`::moving from state init -> state preparing >Thread-338::INFO::2014-05-13 12:57:14,002::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='aa069361-8e06-4459-a17b-748c0a2a828e', options=None) >Thread-338::DEBUG::2014-05-13 12:57:14,003::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1052df76-9358-43c9-8411-2e9e1b6930f7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2572' at 'getStoragePoolInfo' >Thread-338::DEBUG::2014-05-13 12:57:14,003::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' for lock type 'shared' >Thread-338::DEBUG::2014-05-13 12:57:14,003::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free. Now locking as 'shared' (1 active user) >Thread-338::DEBUG::2014-05-13 12:57:14,003::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.aa069361-8e06-4459-a17b-748c0a2a828e`ReqID=`1052df76-9358-43c9-8411-2e9e1b6930f7`::Granted request >Thread-338::DEBUG::2014-05-13 12:57:14,004::task::827::TaskManager.Task::(resourceAcquired) Task=`6014a953-12af-4c7a-b595-b20be0acecae`::_resourcesAcquired: Storage.aa069361-8e06-4459-a17b-748c0a2a828e (shared) >Thread-338::DEBUG::2014-05-13 12:57:14,004::task::990::TaskManager.Task::(_decref) Task=`6014a953-12af-4c7a-b595-b20be0acecae`::ref 1 aborting False >Thread-338::INFO::2014-05-13 12:57:14,005::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:57:14,005::task::1185::TaskManager.Task::(prepare) Task=`6014a953-12af-4c7a-b595-b20be0acecae`::finished: {'info': {'name': 'DC_2', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': u'208946c2-5db1-4ef4-80b1-9aeb37f2905e:Active,41830dad-f012-4b4d-b636-863f66d5ca4b:Active,572b0496-5eab-41cd-a976-4514f111898e:Active', 'master_uuid': '41830dad-f012-4b4d-b636-863f66d5ca4b', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'208946c2-5db1-4ef4-80b1-9aeb37f2905e': {'status': u'Active', 'diskfree': '9597616128', 'isoprefix': '/rhev/data-center/mnt/dhcp163-66.scl.lab.tlv.redhat.com:_var_lib_exports_iso-20140512085215/208946c2-5db1-4ef4-80b1-9aeb37f2905e/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '14870904832', 'version': 0}, u'41830dad-f012-4b4d-b636-863f66d5ca4b': {'status': u'Active', 'diskfree': '155692564480', 'isoprefix': '', 'alerts': [], 'disktotal': '159853314048', 'version': 3}, u'572b0496-5eab-41cd-a976-4514f111898e': {'status': u'Active', 'diskfree': '1923393191936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246071808', 'version': 3}}} >Thread-338::DEBUG::2014-05-13 12:57:14,005::task::595::TaskManager.Task::(_updateState) Task=`6014a953-12af-4c7a-b595-b20be0acecae`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-05-13 12:57:14,005::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.aa069361-8e06-4459-a17b-748c0a2a828e': < ResourceRef 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', isValid: 'True' obj: 'None'>} >Thread-338::DEBUG::2014-05-13 12:57:14,005::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-05-13 12:57:14,006::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' >Thread-338::DEBUG::2014-05-13 12:57:14,006::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' (0 active users) >Thread-338::DEBUG::2014-05-13 12:57:14,006::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e' is free, finding out if anyone is waiting for it. >Thread-338::DEBUG::2014-05-13 12:57:14,006::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.aa069361-8e06-4459-a17b-748c0a2a828e', Clearing records. >Thread-338::DEBUG::2014-05-13 12:57:14,006::task::990::TaskManager.Task::(_decref) Task=`6014a953-12af-4c7a-b595-b20be0acecae`::ref 0 aborting False >Thread-344::DEBUG::2014-05-13 12:57:14,060::fileSD::259::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.35.160.108:_RHEV_ogofen_3/572b0496-5eab-41cd-a976-4514f111898e/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-344::DEBUG::2014-05-13 12:57:14,070::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n348 bytes (348 B) copied, 0.000452625 s, 769 kB/s\n'; <rc> = 0 >Dummy-337::DEBUG::2014-05-13 12:57:15,254::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:57:15,335::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.038502 s, 26.6 MB/s\n'; <rc> = 0 >BindingXMLRPC::ERROR::2014-05-13 12:57:15,820::BindingXMLRPC::84::vds::(threaded_start) xml-rpc handler exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 80, in threaded_start > self.server.handle_request() > File "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request > self._handle_request_noblock() > File "/usr/lib64/python2.6/SocketServer.py", line 288, in _handle_request_noblock > request, client_address = self.get_request() > File "/usr/lib64/python2.6/SocketServer.py", line 456, in get_request > return self.socket.accept() > File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 136, in accept > raise SSL.SSLError("%s, client %s" % (e, address[0])) >SSLError: sslv3 alert certificate unknown, client 10.35.160.208 >Dummy-337::DEBUG::2014-05-13 12:57:17,348::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/aa069361-8e06-4459-a17b-748c0a2a828e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) >Dummy-337::DEBUG::2014-05-13 12:57:17,428::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0325206 s, 31.5 MB/s\n'; <rc> = 0
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Raw
Actions:
View
Attachments on
bug 1084789
:
893231
| 895081