Login
[x]
Log in using an account from:
Fedora Account System
Red Hat Associate
Red Hat Customer
Or login using a Red Hat Bugzilla account
Forgot Password
Login:
Hide Forgot
Create an Account
Red Hat Bugzilla – Attachment 944066 Details for
Bug 1149517
Host monitoring might lag when using jsonrpc
[?]
New
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
|
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh83 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
This site requires JavaScript to be enabled to function correctly, please enable it.
vdsm log when using jsonrpc
vdsm-jsonrpc.log (text/plain), 456.94 KB, created by
Nir Soffer
on 2014-10-05 18:53:21 UTC
(
hide
)
Description:
vdsm log when using jsonrpc
Filename:
MIME Type:
Creator:
Nir Soffer
Created:
2014-10-05 18:53:21 UTC
Size:
456.94 KB
patch
obsolete
>MainThread::INFO::2014-10-05 21:15:02,986::vdsm::131::vds::(run) (PID: 6643) I am the actual vdsm 4.16.6-0.el6 voodoo1.tlv.redhat.com (2.6.32-431.29.2.el6.x86_64) >MainThread::DEBUG::2014-10-05 21:15:02,986::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2014-10-05 21:15:02,986::threadPool::35::Storage.ThreadPool::(__init__) Enter - numThreads: 10, waitTimeout: 3, maxTasks: 500 >MainThread::DEBUG::2014-10-05 21:15:02,988::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt >MainThread::WARNING::2014-10-05 21:15:02,988::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt already exists >MainThread::DEBUG::2014-10-05 21:15:02,991::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/04511d19-f03a-469a-9e31-ab51d6e75a56/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:15:02,991::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/67ba7199-44ec-4562-9c17-ebc3903c7c7f/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:15:02,992::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/83d26f74-2a29-4025-8210-5ab61cb49a82/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:15:02,992::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/fd98c9a9-23f9-43f5-8e7b-4022f5bdd683/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:15:02,993::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/b9c2b835-cdbd-48ea-8d69-7a33042c79b1/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:15:02,993::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/c175b488-c93a-4dff-b709-b137853eab67/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:15:02,993::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/6ca7f5aa-6229-4549-9c6f-54ec7b7099a6/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:15:02,994::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/681bd5bc-4d98-4da3-bd9a-cd37eec7b9af/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:15:02,994::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:15:03,055::supervdsm::77::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2014-10-05 21:15:03,068::multipath::142::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2014-10-05 21:15:03,068::hsm::418::Storage.Misc.excCmd::(__validateLvmLockingType) /usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type (cwd None) >MainThread::DEBUG::2014-10-05 21:15:03,161::hsm::418::Storage.Misc.excCmd::(__validateLvmLockingType) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2014-10-05 21:15:03,161::hsm::442::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >MainThread::DEBUG::2014-10-05 21:15:03,162::hsm::474::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >MainThread::DEBUG::2014-10-05 21:15:03,162::hsm::475::Storage.HSM::(__cleanStorageRepository) Mount list: [] >MainThread::DEBUG::2014-10-05 21:15:03,162::hsm::477::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >MainThread::DEBUG::2014-10-05 21:15:03,164::hsm::520::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >storageRefresh::DEBUG::2014-10-05 21:15:03,164::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2014-10-05 21:15:03,166::dispatcher::46::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >storageRefresh::DEBUG::2014-10-05 21:15:03,166::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >storageRefresh::DEBUG::2014-10-05 21:15:03,170::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >storageRefresh::DEBUG::2014-10-05 21:15:03,179::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >storageRefresh::DEBUG::2014-10-05 21:15:03,180::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >storageRefresh::DEBUG::2014-10-05 21:15:03,182::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >MainThread::DEBUG::2014-10-05 21:15:03,214::task::595::Storage.TaskManager.Task::(_updateState) Task=`fdc0a92a-1b0d-4a01-a3b7-aef7d3f15c34`::moving from state init -> state preparing >MainThread::INFO::2014-10-05 21:15:03,214::logUtils::44::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback(callbackFunc=<bound method clientIF.contEIOVms of <clientIF.clientIF object at 0x1ccc690>>) >MainThread::INFO::2014-10-05 21:15:03,214::logUtils::47::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback, Return response: None >MainThread::DEBUG::2014-10-05 21:15:03,214::task::1191::Storage.TaskManager.Task::(prepare) Task=`fdc0a92a-1b0d-4a01-a3b7-aef7d3f15c34`::finished: None >MainThread::DEBUG::2014-10-05 21:15:03,214::task::595::Storage.TaskManager.Task::(_updateState) Task=`fdc0a92a-1b0d-4a01-a3b7-aef7d3f15c34`::moving from state preparing -> state finished >MainThread::DEBUG::2014-10-05 21:15:03,214::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2014-10-05 21:15:03,214::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2014-10-05 21:15:03,214::task::993::Storage.TaskManager.Task::(_decref) Task=`fdc0a92a-1b0d-4a01-a3b7-aef7d3f15c34`::ref 0 aborting False >MainThread::INFO::2014-10-05 21:15:03,215::momIF::49::MOM::(__init__) Starting up MOM >MainThread::INFO::2014-10-05 21:15:03,218::vmchannels::192::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2014-10-05 21:15:03,220::vmchannels::175::vds::(run) Starting VM channels listener thread. >clientIFinit::DEBUG::2014-10-05 21:15:03,222::libvirtconnection::150::root::(get) trying to connect libvirt >MainThread::DEBUG::2014-10-05 21:15:03,231::protocoldetector::131::vds.MultiProtocolAcceptor::(add_detector) adding detector: <rpc.BindingXMLRPC.XmlDetector instance at 0x1dc5878> >MainThread::DEBUG::2014-10-05 21:15:03,285::protocoldetector::131::vds.MultiProtocolAcceptor::(add_detector) adding detector: <yajsonrpc.stompReactor.StompDetector instance at 0x1f50ea8> >JsonRpcServer::DEBUG::2014-10-05 21:15:03,287::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Detector thread::DEBUG::2014-10-05 21:15:03,293::protocoldetector::70::vds.MultiProtocolAcceptor::(serve_forever) Acceptor running >Detector thread::DEBUG::2014-10-05 21:15:03,294::protocoldetector::72::vds.MultiProtocolAcceptor::(serve_forever) Using required_size=7 >storageRefresh::DEBUG::2014-10-05 21:15:05,194::misc::751::Storage.SamplingMethod::(__call__) Returning last result >storageRefresh::DEBUG::2014-10-05 21:15:05,196::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >storageRefresh::DEBUG::2014-10-05 21:15:05,210::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >storageRefresh::DEBUG::2014-10-05 21:15:05,210::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >storageRefresh::DEBUG::2014-10-05 21:15:05,210::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-10-05 21:15:05,211::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >storageRefresh::DEBUG::2014-10-05 21:15:05,211::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-10-05 21:15:05,211::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >storageRefresh::DEBUG::2014-10-05 21:15:05,211::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-10-05 21:15:05,211::misc::751::Storage.SamplingMethod::(__call__) Returning last result >storageRefresh::DEBUG::2014-10-05 21:15:05,211::lvm::317::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >storageRefresh::DEBUG::2014-10-05 21:15:05,213::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm pvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060007|/dev/mapper/3600144f09dbd0500000053a85076000a|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size (cwd None) >storageRefresh::DEBUG::2014-10-05 21:15:05,292::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' /dev/mapper/3600144f09dbd0500000053a85076000a: read failed after 0 of 4096 at 10737352704: Input/output error\n /dev/mapper/3600144f09dbd0500000053a85076000a: read failed after 0 of 4096 at 10737410048: Input/output error\n /dev/mapper/3600144f09dbd0500000053a85076000a: read failed after 0 of 4096 at 0: Input/output error\n WARNING: Error counts reached a limit of 3. Device /dev/mapper/3600144f09dbd0500000053a85076000a was disabled\n /dev/mapper/1IET_00060007: read failed after 0 of 4096 at 21474770944: Input/output error\n /dev/mapper/1IET_00060007: read failed after 0 of 4096 at 21474828288: Input/output error\n /dev/mapper/1IET_00060007: read failed after 0 of 4096 at 0: Input/output error\n WARNING: Error counts reached a limit of 3. Device /dev/mapper/1IET_00060007 was disabled\n'; <rc> = 0 >storageRefresh::DEBUG::2014-10-05 21:15:05,292::lvm::342::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >storageRefresh::DEBUG::2014-10-05 21:15:05,292::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >storageRefresh::DEBUG::2014-10-05 21:15:05,292::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060007|/dev/mapper/3600144f09dbd0500000053a85076000a|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >storageRefresh::DEBUG::2014-10-05 21:15:05,367::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' /dev/mapper/3600144f09dbd0500000053a85076000a: read failed after 0 of 4096 at 10737352704: Input/output error\n /dev/mapper/3600144f09dbd0500000053a85076000a: read failed after 0 of 4096 at 10737410048: Input/output error\n /dev/mapper/3600144f09dbd0500000053a85076000a: read failed after 0 of 4096 at 0: Input/output error\n WARNING: Error counts reached a limit of 3. Device /dev/mapper/3600144f09dbd0500000053a85076000a was disabled\n /dev/mapper/1IET_00060007: read failed after 0 of 4096 at 21474770944: Input/output error\n /dev/mapper/1IET_00060007: read failed after 0 of 4096 at 21474828288: Input/output error\n /dev/mapper/1IET_00060007: read failed after 0 of 4096 at 0: Input/output error\n WARNING: Error counts reached a limit of 3. Device /dev/mapper/1IET_00060007 was disabled\n No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-10-05 21:15:05,367::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >storageRefresh::DEBUG::2014-10-05 21:15:05,367::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060007|/dev/mapper/3600144f09dbd0500000053a85076000a|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags (cwd None) >storageRefresh::DEBUG::2014-10-05 21:15:05,441::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' /dev/mapper/3600144f09dbd0500000053a85076000a: read failed after 0 of 4096 at 10737352704: Input/output error\n /dev/mapper/3600144f09dbd0500000053a85076000a: read failed after 0 of 4096 at 10737410048: Input/output error\n /dev/mapper/3600144f09dbd0500000053a85076000a: read failed after 0 of 4096 at 0: Input/output error\n WARNING: Error counts reached a limit of 3. Device /dev/mapper/3600144f09dbd0500000053a85076000a was disabled\n /dev/mapper/1IET_00060007: read failed after 0 of 4096 at 21474770944: Input/output error\n /dev/mapper/1IET_00060007: read failed after 0 of 4096 at 21474828288: Input/output error\n /dev/mapper/1IET_00060007: read failed after 0 of 4096 at 0: Input/output error\n WARNING: Error counts reached a limit of 3. Device /dev/mapper/1IET_00060007 was disabled\n No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-10-05 21:15:05,441::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >storageRefresh::DEBUG::2014-10-05 21:15:05,441::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060007|/dev/mapper/3600144f09dbd0500000053a85076000a|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >storageRefresh::DEBUG::2014-10-05 21:15:05,514::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' /dev/mapper/3600144f09dbd0500000053a85076000a: read failed after 0 of 4096 at 10737352704: Input/output error\n /dev/mapper/3600144f09dbd0500000053a85076000a: read failed after 0 of 4096 at 10737410048: Input/output error\n /dev/mapper/3600144f09dbd0500000053a85076000a: read failed after 0 of 4096 at 0: Input/output error\n WARNING: Error counts reached a limit of 3. Device /dev/mapper/3600144f09dbd0500000053a85076000a was disabled\n /dev/mapper/1IET_00060007: read failed after 0 of 4096 at 21474770944: Input/output error\n /dev/mapper/1IET_00060007: read failed after 0 of 4096 at 21474828288: Input/output error\n /dev/mapper/1IET_00060007: read failed after 0 of 4096 at 0: Input/output error\n WARNING: Error counts reached a limit of 3. Device /dev/mapper/1IET_00060007 was disabled\n No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-10-05 21:15:05,515::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >storageRefresh::DEBUG::2014-10-05 21:15:05,515::hsm::387::Storage.HSM::(storageRefresh) HSM is ready >Detector thread::DEBUG::2014-10-05 21:17:00,579::protocoldetector::166::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.201.227:58466 >Detector thread::DEBUG::2014-10-05 21:17:00,662::protocoldetector::177::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.201.227:58466 >Detector thread::DEBUG::2014-10-05 21:17:00,662::protocoldetector::203::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol stomp from 10.35.201.227:58466 >Detector thread::DEBUG::2014-10-05 21:17:00,662::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='CONNECT'> >Detector thread::INFO::2014-10-05 21:17:00,662::stompReactor::66::Broker.StompAdapter::(_cmd_connect) Processing CONNECT request >Detector thread::DEBUG::2014-10-05 21:17:00,662::stompReactor::338::protocoldetector.StompDetector::(handleSocket) Stomp detected from ('10.35.201.227', 58466) >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:17:00,747::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SUBSCRIBE'> >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:17:00,747::stompReactor::89::Broker.StompAdapter::(_cmd_subscribe) Subscribe command ignored >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:17:00,867::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-13::DEBUG::2014-10-05 21:17:00,868::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getCapabilities' in bridge with {} >JsonRpcServer::DEBUG::2014-10-05 21:17:00,868::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-13::DEBUG::2014-10-05 21:17:00,905::utils::738::root::(execCmd) /sbin/ip route show to 0.0.0.0/0 table all (cwd None) >Thread-13::DEBUG::2014-10-05 21:17:00,911::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:17:00,924::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/bonding_masters (cwd None) >Thread-13::DEBUG::2014-10-05 21:17:00,934::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:17:00,935::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/0F0VIVHEV7xcb4a/bonding/mode (cwd None) >Thread-13::DEBUG::2014-10-05 21:17:00,942::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:17:00,943::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/0F0VIVHEV7xcb4a/bonding/mode (cwd None) >Thread-13::DEBUG::2014-10-05 21:17:00,950::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:17:00,951::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/0F0VIVHEV7xcb4a/bonding/mode (cwd None) >Thread-13::DEBUG::2014-10-05 21:17:00,959::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:17:00,959::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/0F0VIVHEV7xcb4a/bonding/mode (cwd None) >Thread-13::DEBUG::2014-10-05 21:17:00,967::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:17:00,968::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/0F0VIVHEV7xcb4a/bonding/mode (cwd None) >Thread-13::DEBUG::2014-10-05 21:17:00,976::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:17:00,977::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/0F0VIVHEV7xcb4a/bonding/mode (cwd None) >Thread-13::DEBUG::2014-10-05 21:17:00,987::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:17:00,988::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/0F0VIVHEV7xcb4a/bonding/mode (cwd None) >Thread-13::DEBUG::2014-10-05 21:17:00,996::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:17:00,997::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/bonding_masters (cwd None) >Thread-13::DEBUG::2014-10-05 21:17:01,027::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:17:01,042::caps::646::root::(get) VirtioRNG DISABLED: libvirt version 0.10.2-29.el6_5.12 required >= 0.10.2-31 >Thread-13::DEBUG::2014-10-05 21:17:01,042::utils::738::root::(execCmd) numactl --hardware (cwd None) >Thread-13::DEBUG::2014-10-05 21:17:01,046::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:17:01,047::utils::738::root::(execCmd) sysctl -n -e kernel.numa_balancing (cwd None) >Thread-13::DEBUG::2014-10-05 21:17:01,050::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:17:01,053::caps::269::root::(getLiveMergeSupport) libvirt is missing 'VIR_DOMAIN_BLOCK_COMMIT_RELATIVE': live merge disabled >Thread-13::DEBUG::2014-10-05 21:17:01,060::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.getCapabilities' in bridge with {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:nsoffer1'}], 'FC': []}, 'packages2': {'kernel': {'release': '431.29.2.el6.x86_64', 'buildtime': 1406465746.0, 'version': '2.6.32'}, 'spice-server': {'release': '6.el6_5.2', 'buildtime': 1401730350L, 'version': '0.12.4'}, 'vdsm': {'release': '0.el6', 'buildtime': 1412232469L, 'version': '4.16.6'}, 'qemu-kvm': {'release': '2.415.el6_5.14', 'buildtime': 1408575896L, 'version': '0.12.1.2'}, 'libvirt': {'release': '29.el6_5.12', 'buildtime': 1408610144L, 'version': '0.10.2'}, 'qemu-img': {'release': '2.415.el6_5.14', 'buildtime': 1408575896L, 'version': '0.12.1.2'}, 'mom': {'release': '0.0.master.20140911.gitcf7ff17.el6', 'buildtime': 1410462635L, 'version': '0.4.2'}}, 'numaNodeDistance': {'0': [10]}, 'cpuModel': 'Intel Core Processor (Haswell)', 'liveMerge': 'false', 'hooks': {}, 'cpuSockets': '8', 'vmTypes': ['kvm'], 'selinux': {'mode': '1'}, 'kdumpStatus': 1, 'supportedProtocols': ['2.2', '2.3'], 'networks': {'ovirtmgmt': {'iface': 'ovirtmgmt', 'addr': '10.35.0.100', 'cfg': {'DEFROUTE': 'yes', 'HOTPLUG': 'no', 'MTU': '1500', 'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'STP': 'off', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'bridged': True, 'ipv6addrs': ['fe80::5054:ff:fe87:196f/64'], 'gateway': '10.35.1.254', 'bootproto4': 'dhcp', 'netmask': '255.255.254.0', 'stp': 'off', 'ipv4addrs': ['10.35.0.100/23'], 'mtu': '1500', 'ipv6gateway': '::', 'ports': ['eth0']}}, 'bridges': {'ovirtmgmt': {'addr': '10.35.0.100', 'cfg': {'DEFROUTE': 'yes', 'HOTPLUG': 'no', 'MTU': '1500', 'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'STP': 'off', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'ipv6addrs': ['fe80::5054:ff:fe87:196f/64'], 'mtu': '1500', 'netmask': '255.255.254.0', 'stp': 'off', 'ipv4addrs': ['10.35.0.100/23'], 'ipv6gateway': '::', 'gateway': '10.35.1.254', 'opts': {'topology_change_detected': '0', 'multicast_last_member_count': '2', 'hash_elasticity': '4', 'multicast_query_response_interval': '999', 'multicast_snooping': '1', 'multicast_startup_query_interval': '3124', 'hello_timer': '151', 'multicast_querier_interval': '25496', 'max_age': '1999', 'hash_max': '512', 'stp_state': '0', 'root_id': '8000.52540087196f', 'priority': '32768', 'multicast_membership_interval': '25996', 'root_path_cost': '0', 'root_port': '0', 'multicast_querier': '0', 'multicast_startup_query_count': '2', 'hello_time': '199', 'topology_change': '0', 'bridge_id': '8000.52540087196f', 'topology_change_timer': '0', 'ageing_time': '29995', 'gc_timer': '51', 'group_addr': '1:80:c2:0:0:0', 'tcn_timer': '0', 'multicast_query_interval': '12498', 'multicast_last_member_interval': '99', 'multicast_router': '1', 'forward_delay': '0'}, 'ports': ['eth0']}}, 'uuid': '85CFDED0-FD24-4645-AF7C-D47A088AE9E8', 'lastClientIface': 'ovirtmgmt', 'nics': {'eth0': {'addr': '', 'cfg': {'BRIDGE': 'ovirtmgmt', 'NM_CONTROLLED': 'no', 'MTU': '1500', 'HWADDR': '52:54:00:87:19:6f', 'DEVICE': 'eth0', 'ONBOOT': 'yes'}, 'ipv6addrs': [], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': '52:54:00:87:19:6f', 'speed': 0}}, 'software_revision': '0', 'clusterLevels': ['3.0', '3.1', '3.2', '3.3', '3.4', '3.5'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,mmx,fxsr,sse,sse2,ss,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,rep_good,unfair_spinlock,pni,pclmulqdq,vmx,ssse3,fma,cx16,pcid,sse4_1,sse4_2,x2apic,movbe,popcnt,tsc_deadline_timer,aes,xsave,avx,f16c,rdrand,hypervisor,lahf_lm,abm,xsaveopt,vnmi,ept,fsgsbase,bmi1,hle,avx2,smep,bmi2,erms,invpcid,rtm,model_Haswell,model_Nehalem,model_Conroe,model_Penryn,model_Westmere,model_SandyBridge', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:nsoffer1', 'netConfigDirty': 'False', 'supportedENGINEs': ['3.0', '3.1', '3.2', '3.3', '3.4', '3.5'], 'autoNumaBalancing': 2, 'reservedMem': '321', 'bondings': {'bond4': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond0': {'addr': '', 'cfg': {}, 'ipv6addrs': [], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': '00:00:00:00:00:00', 'slaves': [], 'opts': {}}, 'bond1': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond2': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond3': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}}, 'software_version': '4.16', 'memSize': '3734', 'cpuSpeed': '3392.160', 'numaNodes': {u'0': {'totalMemory': '3734', 'cpus': [0, 1, 2, 3, 4, 5, 6, 7]}}, 'version_name': 'Snow Man', 'vlans': {}, 'cpuCores': '8', 'kvmEnabled': 'true', 'guestOverhead': '65', 'cpuThreads': '8', 'emulatedMachines': [u'rhel6.5.0', u'pc', u'rhel6.4.0', u'rhel6.3.0', u'rhel6.2.0', u'rhel6.1.0', u'rhel6.0.0', u'rhel5.5.0', u'rhel5.4.4', u'rhel5.4.0'], 'operatingSystem': {'release': '6.5.0.1.el6', 'version': '6Server', 'name': 'RHEL'}} >Thread-13::DEBUG::2014-10-05 21:17:01,062::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:17:01,159::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-14::DEBUG::2014-10-05 21:17:01,160::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getHardwareInfo' in bridge with {} >JsonRpcServer::DEBUG::2014-10-05 21:17:01,160::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-14::DEBUG::2014-10-05 21:17:01,167::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.getHardwareInfo' in bridge with {'systemProductName': 'Bochs', 'systemUUID': '85cfded0-fd24-4645-af7c-d47a088ae9e8', 'systemManufacturer': 'Bochs'} >Thread-14::DEBUG::2014-10-05 21:17:01,167::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:17:01,315::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-15::DEBUG::2014-10-05 21:17:01,316::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getCapabilities' in bridge with {} >JsonRpcServer::DEBUG::2014-10-05 21:17:01,316::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-15::DEBUG::2014-10-05 21:17:01,317::utils::738::root::(execCmd) /sbin/ip route show to 0.0.0.0/0 table all (cwd None) >Thread-15::DEBUG::2014-10-05 21:17:01,321::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2014-10-05 21:17:01,334::caps::646::root::(get) VirtioRNG DISABLED: libvirt version 0.10.2-29.el6_5.12 required >= 0.10.2-31 >Thread-15::DEBUG::2014-10-05 21:17:01,335::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.getCapabilities' in bridge with {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:nsoffer1'}], 'FC': []}, 'packages2': {'kernel': {'release': '431.29.2.el6.x86_64', 'buildtime': 1406465746.0, 'version': '2.6.32'}, 'spice-server': {'release': '6.el6_5.2', 'buildtime': 1401730350L, 'version': '0.12.4'}, 'vdsm': {'release': '0.el6', 'buildtime': 1412232469L, 'version': '4.16.6'}, 'qemu-kvm': {'release': '2.415.el6_5.14', 'buildtime': 1408575896L, 'version': '0.12.1.2'}, 'libvirt': {'release': '29.el6_5.12', 'buildtime': 1408610144L, 'version': '0.10.2'}, 'qemu-img': {'release': '2.415.el6_5.14', 'buildtime': 1408575896L, 'version': '0.12.1.2'}, 'mom': {'release': '0.0.master.20140911.gitcf7ff17.el6', 'buildtime': 1410462635L, 'version': '0.4.2'}}, 'numaNodeDistance': {'0': [10]}, 'cpuModel': 'Intel Core Processor (Haswell)', 'liveMerge': 'false', 'hooks': {}, 'cpuSockets': '8', 'vmTypes': ['kvm'], 'selinux': {'mode': '1'}, 'kdumpStatus': 1, 'supportedProtocols': ['2.2', '2.3'], 'networks': {'ovirtmgmt': {'iface': 'ovirtmgmt', 'addr': '10.35.0.100', 'cfg': {'DEFROUTE': 'yes', 'HOTPLUG': 'no', 'MTU': '1500', 'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'STP': 'off', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'bridged': True, 'ipv6addrs': ['fe80::5054:ff:fe87:196f/64'], 'gateway': '10.35.1.254', 'bootproto4': 'dhcp', 'netmask': '255.255.254.0', 'stp': 'off', 'ipv4addrs': ['10.35.0.100/23'], 'mtu': '1500', 'ipv6gateway': '::', 'ports': ['eth0']}}, 'bridges': {'ovirtmgmt': {'addr': '10.35.0.100', 'cfg': {'DEFROUTE': 'yes', 'HOTPLUG': 'no', 'MTU': '1500', 'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'STP': 'off', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'ipv6addrs': ['fe80::5054:ff:fe87:196f/64'], 'mtu': '1500', 'netmask': '255.255.254.0', 'stp': 'off', 'ipv4addrs': ['10.35.0.100/23'], 'ipv6gateway': '::', 'gateway': '10.35.1.254', 'opts': {'topology_change_detected': '0', 'multicast_last_member_count': '2', 'hash_elasticity': '4', 'multicast_query_response_interval': '999', 'multicast_snooping': '1', 'multicast_startup_query_interval': '3124', 'hello_timer': '121', 'multicast_querier_interval': '25496', 'max_age': '1999', 'hash_max': '512', 'stp_state': '0', 'root_id': '8000.52540087196f', 'priority': '32768', 'multicast_membership_interval': '25996', 'root_path_cost': '0', 'root_port': '0', 'multicast_querier': '0', 'multicast_startup_query_count': '2', 'hello_time': '199', 'topology_change': '0', 'bridge_id': '8000.52540087196f', 'topology_change_timer': '0', 'ageing_time': '29995', 'gc_timer': '21', 'group_addr': '1:80:c2:0:0:0', 'tcn_timer': '0', 'multicast_query_interval': '12498', 'multicast_last_member_interval': '99', 'multicast_router': '1', 'forward_delay': '0'}, 'ports': ['eth0']}}, 'uuid': '85CFDED0-FD24-4645-AF7C-D47A088AE9E8', 'lastClientIface': 'ovirtmgmt', 'nics': {'eth0': {'addr': '', 'cfg': {'BRIDGE': 'ovirtmgmt', 'NM_CONTROLLED': 'no', 'MTU': '1500', 'HWADDR': '52:54:00:87:19:6f', 'DEVICE': 'eth0', 'ONBOOT': 'yes'}, 'ipv6addrs': [], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': '52:54:00:87:19:6f', 'speed': 0}}, 'software_revision': '0', 'clusterLevels': ['3.0', '3.1', '3.2', '3.3', '3.4', '3.5'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,mmx,fxsr,sse,sse2,ss,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,rep_good,unfair_spinlock,pni,pclmulqdq,vmx,ssse3,fma,cx16,pcid,sse4_1,sse4_2,x2apic,movbe,popcnt,tsc_deadline_timer,aes,xsave,avx,f16c,rdrand,hypervisor,lahf_lm,abm,xsaveopt,vnmi,ept,fsgsbase,bmi1,hle,avx2,smep,bmi2,erms,invpcid,rtm,model_Haswell,model_Nehalem,model_Conroe,model_Penryn,model_Westmere,model_SandyBridge', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:nsoffer1', 'netConfigDirty': 'False', 'supportedENGINEs': ['3.0', '3.1', '3.2', '3.3', '3.4', '3.5'], 'autoNumaBalancing': 2, 'reservedMem': '321', 'bondings': {'bond4': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond0': {'addr': '', 'cfg': {}, 'ipv6addrs': [], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': '00:00:00:00:00:00', 'slaves': [], 'opts': {}}, 'bond1': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond2': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond3': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}}, 'software_version': '4.16', 'memSize': '3734', 'cpuSpeed': '3392.160', 'numaNodes': {u'0': {'totalMemory': '3734', 'cpus': [0, 1, 2, 3, 4, 5, 6, 7]}}, 'version_name': 'Snow Man', 'vlans': {}, 'cpuCores': '8', 'kvmEnabled': 'true', 'guestOverhead': '65', 'cpuThreads': '8', 'emulatedMachines': [u'rhel6.5.0', u'pc', u'rhel6.4.0', u'rhel6.3.0', u'rhel6.2.0', u'rhel6.1.0', u'rhel6.0.0', u'rhel5.5.0', u'rhel5.4.4', u'rhel5.4.0'], 'operatingSystem': {'release': '6.5.0.1.el6', 'version': '6Server', 'name': 'RHEL'}} >Thread-15::DEBUG::2014-10-05 21:17:01,336::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:17:01,433::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-16::DEBUG::2014-10-05 21:17:01,433::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getHardwareInfo' in bridge with {} >JsonRpcServer::DEBUG::2014-10-05 21:17:01,433::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-16::DEBUG::2014-10-05 21:17:01,435::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.getHardwareInfo' in bridge with {'systemProductName': 'Bochs', 'systemUUID': '85cfded0-fd24-4645-af7c-d47a088ae9e8', 'systemManufacturer': 'Bochs'} >Thread-16::DEBUG::2014-10-05 21:17:01,435::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:17:01,554::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:17:01,555::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-17::DEBUG::2014-10-05 21:17:01,556::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:17:01,745::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-18::DEBUG::2014-10-05 21:17:01,746::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.connectStorageServer' in bridge with {u'connectionParams': [{u'id': u'af150e8e-4d74-4810-b8bf-0c6a19637dbf', u'connection': u'10.35.16.25', u'iqn': u'iqn.1994-05.com.redhat:nsoffer-target1', u'user': u'', u'tpgt': u'1', u'password': u'', u'port': u'3260'}, {u'id': u'e8d58651-6414-4fc2-b281-2141cc97829b', u'connection': u'10.35.0.98', u'iqn': u'iqn.1994-05.com.redhat:target6', u'user': u'', u'tpgt': u'1', u'password': u'', u'port': u'3260'}, {u'id': u'8f399b5f-1237-479e-82e6-eedb202b66fd', u'connection': u'10.35.16.25', u'iqn': u'iqn.1994-05.com.redhat:nsoffer-target2', u'user': u'', u'tpgt': u'1', u'password': u'', u'port': u'3260'}], u'storagepoolID': u'b86b687a-d073-497a-ac8a-249025419a3e', u'domainType': 3} >JsonRpcServer::DEBUG::2014-10-05 21:17:01,746::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-18::DEBUG::2014-10-05 21:17:01,747::task::595::Storage.TaskManager.Task::(_updateState) Task=`c0eec676-7d58-4d63-88ce-ef6e03413297`::moving from state init -> state preparing >Thread-18::INFO::2014-10-05 21:17:01,747::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=3, spUUID=u'b86b687a-d073-497a-ac8a-249025419a3e', conList=[{u'connection': u'10.35.16.25', u'iqn': u'iqn.1994-05.com.redhat:nsoffer-target1', u'user': u'', u'tpgt': u'1', u'password': '******', u'id': u'af150e8e-4d74-4810-b8bf-0c6a19637dbf', u'port': u'3260'}, {u'connection': u'10.35.0.98', u'iqn': u'iqn.1994-05.com.redhat:target6', u'user': u'', u'tpgt': u'1', u'password': '******', u'id': u'e8d58651-6414-4fc2-b281-2141cc97829b', u'port': u'3260'}, {u'connection': u'10.35.16.25', u'iqn': u'iqn.1994-05.com.redhat:nsoffer-target2', u'user': u'', u'tpgt': u'1', u'password': '******', u'id': u'8f399b5f-1237-479e-82e6-eedb202b66fd', u'port': u'3260'}], options=None) >Thread-18::DEBUG::2014-10-05 21:17:01,748::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:nsoffer-target1 -I default -p 10.35.16.25:3260,1 --op=new (cwd None) >Thread-18::DEBUG::2014-10-05 21:17:01,759::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:17:01,759::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:nsoffer-target1 -I default -p 10.35.16.25:3260,1 -l (cwd None) >Thread-18::DEBUG::2014-10-05 21:17:02,582::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:17:02,584::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:nsoffer-target1 -I default -p 10.35.16.25:3260,1 -n node.startup -v manual --op=update (cwd None) >Thread-18::DEBUG::2014-10-05 21:17:02,630::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:17:02,632::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-18::DEBUG::2014-10-05 21:17:02,634::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060007|/dev/mapper/3600144f09dbd0500000053a85076000a|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-18::DEBUG::2014-10-05 21:17:02,741::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' /dev/mapper/3600144f09dbd0500000053a85076000a: read failed after 0 of 4096 at 10737352704: Input/output error\n /dev/mapper/3600144f09dbd0500000053a85076000a: read failed after 0 of 4096 at 10737410048: Input/output error\n /dev/mapper/3600144f09dbd0500000053a85076000a: read failed after 0 of 4096 at 0: Input/output error\n WARNING: Error counts reached a limit of 3. Device /dev/mapper/3600144f09dbd0500000053a85076000a was disabled\n /dev/mapper/1IET_00060007: read failed after 0 of 4096 at 21474770944: Input/output error\n /dev/mapper/1IET_00060007: read failed after 0 of 4096 at 21474828288: Input/output error\n /dev/mapper/1IET_00060007: read failed after 0 of 4096 at 0: Input/output error\n WARNING: Error counts reached a limit of 3. Device /dev/mapper/1IET_00060007 was disabled\n No volume groups found\n'; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:17:02,743::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-18::DEBUG::2014-10-05 21:17:02,744::hsm::2396::Storage.HSM::(__prefetchDomains) Found SD uuids: () >Thread-18::DEBUG::2014-10-05 21:17:02,745::hsm::2452::Storage.HSM::(connectStorageServer) knownSDs: {} >Thread-18::DEBUG::2014-10-05 21:17:02,746::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:target6 -I default -p 10.35.0.98:3260,1 --op=new (cwd None) >Thread-18::DEBUG::2014-10-05 21:17:02,762::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:17:02,764::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:target6 -I default -p 10.35.0.98:3260,1 -l (cwd None) >Thread-18::DEBUG::2014-10-05 21:17:03,350::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:17:03,351::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:target6 -I default -p 10.35.0.98:3260,1 -n node.startup -v manual --op=update (cwd None) >Thread-18::DEBUG::2014-10-05 21:17:03,366::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:17:03,367::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-18::DEBUG::2014-10-05 21:17:03,367::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060007|/dev/mapper/3600144f09dbd0500000053a85076000a|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-18::DEBUG::2014-10-05 21:17:03,618::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = " Couldn't find device with uuid pGRciD-wcLp-X1ME-MijE-P1B9-7XWo-ycTXdG.\n Couldn't find device with uuid 5zWyZx-cpAZ-tMON-3wa5-s6pN-Xj7N-b3fPs5.\n Couldn't find device with uuid c2LMVb-PXdj-XpTE-4f06-2rrh-TdAG-zzehcn.\n Couldn't find device with uuid 1kMage-eAId-qy7G-oIqH-tfbH-ScLD-gqv7JM.\n"; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:17:03,619::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-18::DEBUG::2014-10-05 21:17:03,619::hsm::2396::Storage.HSM::(__prefetchDomains) Found SD uuids: ('c73a3ced-4b41-4b4e-93f7-494d63e7e7ae', 'ff559f46-c495-4f6b-901c-2a624042a050') >Thread-18::DEBUG::2014-10-05 21:17:03,619::hsm::2452::Storage.HSM::(connectStorageServer) knownSDs: {c73a3ced-4b41-4b4e-93f7-494d63e7e7ae: storage.blockSD.findDomain, ff559f46-c495-4f6b-901c-2a624042a050: storage.blockSD.findDomain} >Thread-18::DEBUG::2014-10-05 21:17:03,620::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:nsoffer-target2 -I default -p 10.35.16.25:3260,1 --op=new (cwd None) >Thread-18::DEBUG::2014-10-05 21:17:03,637::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:17:03,637::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:nsoffer-target2 -I default -p 10.35.16.25:3260,1 -l (cwd None) >Thread-18::DEBUG::2014-10-05 21:17:04,427::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:17:04,428::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:nsoffer-target2 -I default -p 10.35.16.25:3260,1 -n node.startup -v manual --op=update (cwd None) >Thread-18::DEBUG::2014-10-05 21:17:04,444::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:17:04,445::hsm::2396::Storage.HSM::(__prefetchDomains) Found SD uuids: ('c73a3ced-4b41-4b4e-93f7-494d63e7e7ae', 'ff559f46-c495-4f6b-901c-2a624042a050') >Thread-18::DEBUG::2014-10-05 21:17:04,445::hsm::2452::Storage.HSM::(connectStorageServer) knownSDs: {c73a3ced-4b41-4b4e-93f7-494d63e7e7ae: storage.blockSD.findDomain, ff559f46-c495-4f6b-901c-2a624042a050: storage.blockSD.findDomain} >Thread-18::INFO::2014-10-05 21:17:04,445::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': u'af150e8e-4d74-4810-b8bf-0c6a19637dbf'}, {'status': 0, 'id': u'e8d58651-6414-4fc2-b281-2141cc97829b'}, {'status': 0, 'id': u'8f399b5f-1237-479e-82e6-eedb202b66fd'}]} >Thread-18::DEBUG::2014-10-05 21:17:04,449::task::1191::Storage.TaskManager.Task::(prepare) Task=`c0eec676-7d58-4d63-88ce-ef6e03413297`::finished: {'statuslist': [{'status': 0, 'id': u'af150e8e-4d74-4810-b8bf-0c6a19637dbf'}, {'status': 0, 'id': u'e8d58651-6414-4fc2-b281-2141cc97829b'}, {'status': 0, 'id': u'8f399b5f-1237-479e-82e6-eedb202b66fd'}]} >Thread-18::DEBUG::2014-10-05 21:17:04,449::task::595::Storage.TaskManager.Task::(_updateState) Task=`c0eec676-7d58-4d63-88ce-ef6e03413297`::moving from state preparing -> state finished >Thread-18::DEBUG::2014-10-05 21:17:04,449::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-10-05 21:17:04,449::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-10-05 21:17:04,450::task::993::Storage.TaskManager.Task::(_decref) Task=`c0eec676-7d58-4d63-88ce-ef6e03413297`::ref 0 aborting False >Thread-18::DEBUG::2014-10-05 21:17:04,450::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.connectStorageServer' in bridge with [{'status': 0, 'id': u'af150e8e-4d74-4810-b8bf-0c6a19637dbf'}, {'status': 0, 'id': u'e8d58651-6414-4fc2-b281-2141cc97829b'}, {'status': 0, 'id': u'8f399b5f-1237-479e-82e6-eedb202b66fd'}] >Thread-18::DEBUG::2014-10-05 21:17:04,450::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:17:04,565::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:17:04,568::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-19::DEBUG::2014-10-05 21:17:04,569::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.connect' in bridge with {u'masterVersion': 5, u'domainDict': {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': u'active', u'ff559f46-c495-4f6b-901c-2a624042a050': u'active'}, u'storagepoolID': u'b86b687a-d073-497a-ac8a-249025419a3e', u'scsiKey': u'b86b687a-d073-497a-ac8a-249025419a3e', u'masterSdUUID': u'ff559f46-c495-4f6b-901c-2a624042a050', u'hostID': 3} >Thread-19::DEBUG::2014-10-05 21:17:04,570::task::595::Storage.TaskManager.Task::(_updateState) Task=`61433763-7cf9-4052-804b-79d9ee63238e`::moving from state init -> state preparing >Thread-19::INFO::2014-10-05 21:17:04,570::logUtils::44::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID=u'b86b687a-d073-497a-ac8a-249025419a3e', hostID=3, msdUUID=u'ff559f46-c495-4f6b-901c-2a624042a050', masterVersion=5, domainsMap={u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': u'active', u'ff559f46-c495-4f6b-901c-2a624042a050': u'active'}, options=None) >Thread-19::DEBUG::2014-10-05 21:17:04,571::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.HsmDomainMonitorLock`ReqID=`780f766e-2253-4172-81cd-691422eaecec`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1011' at 'connectStoragePool' >Thread-19::DEBUG::2014-10-05 21:17:04,571::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.HsmDomainMonitorLock' for lock type 'exclusive' >Thread-19::DEBUG::2014-10-05 21:17:04,572::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.HsmDomainMonitorLock' is free. Now locking as 'exclusive' (1 active user) >Thread-19::DEBUG::2014-10-05 21:17:04,572::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.HsmDomainMonitorLock`ReqID=`780f766e-2253-4172-81cd-691422eaecec`::Granted request >Thread-19::DEBUG::2014-10-05 21:17:04,572::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.b86b687a-d073-497a-ac8a-249025419a3e`ReqID=`38ee33c5-5eb9-4f70-8a60-4f0ca8ca232b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1059' at '_connectStoragePool' >Thread-19::DEBUG::2014-10-05 21:17:04,572::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.b86b687a-d073-497a-ac8a-249025419a3e' for lock type 'exclusive' >Thread-19::DEBUG::2014-10-05 21:17:04,572::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.b86b687a-d073-497a-ac8a-249025419a3e' is free. Now locking as 'exclusive' (1 active user) >Thread-19::DEBUG::2014-10-05 21:17:04,572::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.b86b687a-d073-497a-ac8a-249025419a3e`ReqID=`38ee33c5-5eb9-4f70-8a60-4f0ca8ca232b`::Granted request >Thread-19::DEBUG::2014-10-05 21:17:04,573::spbackends::485::Storage.StoragePoolMemoryBackend::(updateVersionAndDomains) updating domain version to 5 and domains map to {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': u'active', u'ff559f46-c495-4f6b-901c-2a624042a050': u'active'} >Thread-19::INFO::2014-10-05 21:17:04,573::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 5 and domains map {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': u'Active', u'ff559f46-c495-4f6b-901c-2a624042a050': u'Active'} >Thread-19::INFO::2014-10-05 21:17:04,573::sp::133::Storage.StoragePool::(setBackend) updating pool b86b687a-d073-497a-ac8a-249025419a3e backend from type NoneType instance 0x3a11372210 to type StoragePoolMemoryBackend instance 0x7fec10107158 >Thread-19::INFO::2014-10-05 21:17:04,573::sp::631::Storage.StoragePool::(connect) Connect host #3 to the storage pool b86b687a-d073-497a-ac8a-249025419a3e with master domain: ff559f46-c495-4f6b-901c-2a624042a050 (ver = 5) >Thread-19::DEBUG::2014-10-05 21:17:04,573::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-19::DEBUG::2014-10-05 21:17:04,573::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-19::DEBUG::2014-10-05 21:17:04,573::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-19::DEBUG::2014-10-05 21:17:04,573::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-19::DEBUG::2014-10-05 21:17:04,574::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-19::DEBUG::2014-10-05 21:17:04,574::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-19::DEBUG::2014-10-05 21:17:04,574::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-19::DEBUG::2014-10-05 21:17:04,574::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-19::DEBUG::2014-10-05 21:17:04,574::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-19::DEBUG::2014-10-05 21:17:04,574::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-19::DEBUG::2014-10-05 21:17:04,574::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-19::DEBUG::2014-10-05 21:17:04,574::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-19::DEBUG::2014-10-05 21:17:04,774::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-19::DEBUG::2014-10-05 21:17:04,790::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-19::DEBUG::2014-10-05 21:17:04,947::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-19::DEBUG::2014-10-05 21:17:04,947::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-19::DEBUG::2014-10-05 21:17:04,948::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-19::DEBUG::2014-10-05 21:17:04,948::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-19::DEBUG::2014-10-05 21:17:04,948::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-19::DEBUG::2014-10-05 21:17:04,948::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-19::DEBUG::2014-10-05 21:17:04,948::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-19::DEBUG::2014-10-05 21:17:04,948::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-19::DEBUG::2014-10-05 21:17:04,948::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-19::DEBUG::2014-10-05 21:17:04,951::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060001|/dev/mapper/1IET_00060002|/dev/mapper/1IET_00060003|/dev/mapper/1IET_00060004|/dev/mapper/1IET_00060005|/dev/mapper/1IET_00060006|/dev/mapper/1IET_00060007|/dev/mapper/1IET_00060008|/dev/mapper/1IET_00060009|/dev/mapper/1IET_0006000a|/dev/mapper/3600144f09dbd05000000522dde040011|/dev/mapper/3600144f09dbd05000000522dde050012|/dev/mapper/3600144f09dbd05000000522dde070013|/dev/mapper/3600144f09dbd05000000522dde090014|/dev/mapper/3600144f09dbd05000000522dde0a0015|/dev/mapper/3600144f09dbd05000000522dde0c0016|/dev/mapper/3600144f09dbd05000000522dde0d0017|/dev/mapper/3600144f09dbd05000000522dde0f0018|/dev/mapper/3600144f09dbd05000000522dde100019|/dev/mapper/3600144f09dbd05000000522dde12001a|/dev/mapper/3600144f09dbd0500000053a84cba0002|/dev/mapper/3600144f09dbd0500000053a84e730003|/dev/mapper/3600144f09dbd0500000053a84f4d0004|/dev/mapper/3600144f09dbd0500000053a8503b0005|/dev/mapper/3600144f09dbd0500000053a850500006|/dev/mapper/3600144f09dbd0500000053a8505b0007|/dev/mapper/3600144f09dbd0500000053a850640008|/dev/mapper/3600144f09dbd0500000053a8506c0009|/dev/mapper/3600144f09dbd0500000053a85076000a|/dev/mapper/3600144f09dbd0500000053a8507f000b|/dev/mapper/3600144f09dbd0500000053a853ed000c|/dev/mapper/3600144f09dbd0500000053a85401000d|/dev/mapper/3600144f09dbd0500000053a85409000e|/dev/mapper/3600144f09dbd0500000053a85410000f|/dev/mapper/3600144f09dbd0500000053a854180010|/dev/mapper/3600144f09dbd0500000053a854250011|/dev/mapper/3600144f09dbd0500000053a8542d0012|/dev/mapper/3600144f09dbd0500000053a8543e0013|/dev/mapper/3600144f09dbd0500000053a854450014|/dev/mapper/3600144f09dbd0500000053a8544d0015|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name ff559f46-c495-4f6b-901c-2a624042a050 (cwd None) >Thread-19::DEBUG::2014-10-05 21:17:05,171::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-19::DEBUG::2014-10-05 21:17:05,172::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-19::DEBUG::2014-10-05 21:17:05,172::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-19::DEBUG::2014-10-05 21:17:05,172::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata count=1 (cwd None) >Thread-19::DEBUG::2014-10-05 21:17:05,177::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000693796 s, 3.0 MB/s\n'; <rc> = 0 >Thread-19::DEBUG::2014-10-05 21:17:05,177::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000693796 s, 3.0 MB/s'], size: 2048 >Thread-19::DEBUG::2014-10-05 21:17:05,177::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-19::DEBUG::2014-10-05 21:17:05,177::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-19::DEBUG::2014-10-05 21:17:05,177::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-19::DEBUG::2014-10-05 21:17:05,178::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-19::DEBUG::2014-10-05 21:17:05,178::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-19::DEBUG::2014-10-05 21:17:05,178::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-19::DEBUG::2014-10-05 21:17:05,178::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-19::DEBUG::2014-10-05 21:17:05,178::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-19::DEBUG::2014-10-05 21:17:05,178::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060001|/dev/mapper/1IET_00060002|/dev/mapper/1IET_00060003|/dev/mapper/1IET_00060004|/dev/mapper/1IET_00060005|/dev/mapper/1IET_00060006|/dev/mapper/1IET_00060007|/dev/mapper/1IET_00060008|/dev/mapper/1IET_00060009|/dev/mapper/1IET_0006000a|/dev/mapper/3600144f09dbd05000000522dde040011|/dev/mapper/3600144f09dbd05000000522dde050012|/dev/mapper/3600144f09dbd05000000522dde070013|/dev/mapper/3600144f09dbd05000000522dde090014|/dev/mapper/3600144f09dbd05000000522dde0a0015|/dev/mapper/3600144f09dbd05000000522dde0c0016|/dev/mapper/3600144f09dbd05000000522dde0d0017|/dev/mapper/3600144f09dbd05000000522dde0f0018|/dev/mapper/3600144f09dbd05000000522dde100019|/dev/mapper/3600144f09dbd05000000522dde12001a|/dev/mapper/3600144f09dbd0500000053a84cba0002|/dev/mapper/3600144f09dbd0500000053a84e730003|/dev/mapper/3600144f09dbd0500000053a84f4d0004|/dev/mapper/3600144f09dbd0500000053a8503b0005|/dev/mapper/3600144f09dbd0500000053a850500006|/dev/mapper/3600144f09dbd0500000053a8505b0007|/dev/mapper/3600144f09dbd0500000053a850640008|/dev/mapper/3600144f09dbd0500000053a8506c0009|/dev/mapper/3600144f09dbd0500000053a85076000a|/dev/mapper/3600144f09dbd0500000053a8507f000b|/dev/mapper/3600144f09dbd0500000053a853ed000c|/dev/mapper/3600144f09dbd0500000053a85401000d|/dev/mapper/3600144f09dbd0500000053a85409000e|/dev/mapper/3600144f09dbd0500000053a85410000f|/dev/mapper/3600144f09dbd0500000053a854180010|/dev/mapper/3600144f09dbd0500000053a854250011|/dev/mapper/3600144f09dbd0500000053a8542d0012|/dev/mapper/3600144f09dbd0500000053a8543e0013|/dev/mapper/3600144f09dbd0500000053a854450014|/dev/mapper/3600144f09dbd0500000053a8544d0015|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name ff559f46-c495-4f6b-901c-2a624042a050 (cwd None) >Thread-19::DEBUG::2014-10-05 21:17:05,389::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-19::DEBUG::2014-10-05 21:17:05,389::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-19::DEBUG::2014-10-05 21:17:05,389::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'VGUUID=1B1oT5-JcGj-Ru1I-v3Ni-yden-Du9b-I9fSu8', 'CLASS=Data', 'VERSION=3', 'TYPE=ISCSI', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'LOCKPOLICY=', 'SDUUID=ff559f46-c495-4f6b-901c-2a624042a050', 'PHYBLKSIZE=512', u'PV0=pv:1IET_00060007,uuid:QfuzQ5-Ml7u-kFUc-gdmr-rfmS-Aefp-3HODNb,pestart:0,pecount:157,mapoffset:0', 'POOL_UUID=b86b687a-d073-497a-ac8a-249025419a3e', 'ROLE=Master', u'PV1=pv:1IET_00060009,uuid:pGRciD-wcLp-X1ME-MijE-P1B9-7XWo-ycTXdG,pestart:0,pecount:157,mapoffset:157', 'DESCRIPTION=jumbo-el7', '_SHA_CKSUM=1695e16fba6531db32e52743127e50d763522d29'] >Thread-19::DEBUG::2014-10-05 21:17:05,390::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'ff559f46-c495-4f6b-901c-2a624042a050_imageNS' >Thread-19::DEBUG::2014-10-05 21:17:05,390::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'ff559f46-c495-4f6b-901c-2a624042a050_volumeNS' >Thread-19::DEBUG::2014-10-05 21:17:05,391::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'ff559f46-c495-4f6b-901c-2a624042a050_lvmActivationNS' >Thread-19::DEBUG::2014-10-05 21:17:05,391::sp::1390::Storage.StoragePool::(setMasterDomain) Master domain ff559f46-c495-4f6b-901c-2a624042a050 verified, version 5 >Thread-19::DEBUG::2014-10-05 21:17:05,391::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-19::DEBUG::2014-10-05 21:17:05,391::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-19::INFO::2014-10-05 21:17:05,391::domainMonitor::100::Storage.DomainMonitor::(startMonitoring) Start monitoring c73a3ced-4b41-4b4e-93f7-494d63e7e7ae >Thread-19::DEBUG::2014-10-05 21:17:05,391::sp::1413::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `b86b687a-d073-497a-ac8a-249025419a3e` started monitoring domain `c73a3ced-4b41-4b4e-93f7-494d63e7e7ae` >Thread-20::DEBUG::2014-10-05 21:17:05,391::domainMonitor::182::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for c73a3ced-4b41-4b4e-93f7-494d63e7e7ae >Thread-19::INFO::2014-10-05 21:17:05,392::domainMonitor::100::Storage.DomainMonitor::(startMonitoring) Start monitoring ff559f46-c495-4f6b-901c-2a624042a050 >Thread-21::DEBUG::2014-10-05 21:17:05,393::domainMonitor::182::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for ff559f46-c495-4f6b-901c-2a624042a050 >Thread-19::DEBUG::2014-10-05 21:17:05,393::sp::1413::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `b86b687a-d073-497a-ac8a-249025419a3e` started monitoring domain `ff559f46-c495-4f6b-901c-2a624042a050` >Thread-20::DEBUG::2014-10-05 21:17:05,393::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-19::DEBUG::2014-10-05 21:17:05,394::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-19::DEBUG::2014-10-05 21:17:05,395::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e >Thread-20::DEBUG::2014-10-05 21:17:05,395::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060001|/dev/mapper/1IET_00060002|/dev/mapper/1IET_00060003|/dev/mapper/1IET_00060004|/dev/mapper/1IET_00060005|/dev/mapper/1IET_00060006|/dev/mapper/1IET_00060007|/dev/mapper/1IET_00060008|/dev/mapper/1IET_00060009|/dev/mapper/1IET_0006000a|/dev/mapper/3600144f09dbd05000000522dde040011|/dev/mapper/3600144f09dbd05000000522dde050012|/dev/mapper/3600144f09dbd05000000522dde070013|/dev/mapper/3600144f09dbd05000000522dde090014|/dev/mapper/3600144f09dbd05000000522dde0a0015|/dev/mapper/3600144f09dbd05000000522dde0c0016|/dev/mapper/3600144f09dbd05000000522dde0d0017|/dev/mapper/3600144f09dbd05000000522dde0f0018|/dev/mapper/3600144f09dbd05000000522dde100019|/dev/mapper/3600144f09dbd05000000522dde12001a|/dev/mapper/3600144f09dbd0500000053a84cba0002|/dev/mapper/3600144f09dbd0500000053a84e730003|/dev/mapper/3600144f09dbd0500000053a84f4d0004|/dev/mapper/3600144f09dbd0500000053a8503b0005|/dev/mapper/3600144f09dbd0500000053a850500006|/dev/mapper/3600144f09dbd0500000053a8505b0007|/dev/mapper/3600144f09dbd0500000053a850640008|/dev/mapper/3600144f09dbd0500000053a8506c0009|/dev/mapper/3600144f09dbd0500000053a85076000a|/dev/mapper/3600144f09dbd0500000053a8507f000b|/dev/mapper/3600144f09dbd0500000053a853ed000c|/dev/mapper/3600144f09dbd0500000053a85401000d|/dev/mapper/3600144f09dbd0500000053a85409000e|/dev/mapper/3600144f09dbd0500000053a85410000f|/dev/mapper/3600144f09dbd0500000053a854180010|/dev/mapper/3600144f09dbd0500000053a854250011|/dev/mapper/3600144f09dbd0500000053a8542d0012|/dev/mapper/3600144f09dbd0500000053a8543e0013|/dev/mapper/3600144f09dbd0500000053a854450014|/dev/mapper/3600144f09dbd0500000053a8544d0015|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name c73a3ced-4b41-4b4e-93f7-494d63e7e7ae (cwd None) >Thread-21::DEBUG::2014-10-05 21:17:05,395::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgck --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060007|/dev/mapper/1IET_00060009|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' ff559f46-c495-4f6b-901c-2a624042a050 (cwd None) >Thread-19::DEBUG::2014-10-05 21:17:05,414::lvm::365::Storage.OperationMutex::(_reloadvgs) Got the operational mutex >Thread-19::DEBUG::2014-10-05 21:17:05,414::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060001|/dev/mapper/1IET_00060002|/dev/mapper/1IET_00060003|/dev/mapper/1IET_00060004|/dev/mapper/1IET_00060005|/dev/mapper/1IET_00060006|/dev/mapper/1IET_00060007|/dev/mapper/1IET_00060008|/dev/mapper/1IET_00060009|/dev/mapper/1IET_0006000a|/dev/mapper/3600144f09dbd05000000522dde040011|/dev/mapper/3600144f09dbd05000000522dde050012|/dev/mapper/3600144f09dbd05000000522dde070013|/dev/mapper/3600144f09dbd05000000522dde090014|/dev/mapper/3600144f09dbd05000000522dde0a0015|/dev/mapper/3600144f09dbd05000000522dde0c0016|/dev/mapper/3600144f09dbd05000000522dde0d0017|/dev/mapper/3600144f09dbd05000000522dde0f0018|/dev/mapper/3600144f09dbd05000000522dde100019|/dev/mapper/3600144f09dbd05000000522dde12001a|/dev/mapper/3600144f09dbd0500000053a84cba0002|/dev/mapper/3600144f09dbd0500000053a84e730003|/dev/mapper/3600144f09dbd0500000053a84f4d0004|/dev/mapper/3600144f09dbd0500000053a8503b0005|/dev/mapper/3600144f09dbd0500000053a850500006|/dev/mapper/3600144f09dbd0500000053a8505b0007|/dev/mapper/3600144f09dbd0500000053a850640008|/dev/mapper/3600144f09dbd0500000053a8506c0009|/dev/mapper/3600144f09dbd0500000053a85076000a|/dev/mapper/3600144f09dbd0500000053a8507f000b|/dev/mapper/3600144f09dbd0500000053a853ed000c|/dev/mapper/3600144f09dbd0500000053a85401000d|/dev/mapper/3600144f09dbd0500000053a85409000e|/dev/mapper/3600144f09dbd0500000053a85410000f|/dev/mapper/3600144f09dbd0500000053a854180010|/dev/mapper/3600144f09dbd0500000053a854250011|/dev/mapper/3600144f09dbd0500000053a8542d0012|/dev/mapper/3600144f09dbd0500000053a8543e0013|/dev/mapper/3600144f09dbd0500000053a854450014|/dev/mapper/3600144f09dbd0500000053a8544d0015|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name c73a3ced-4b41-4b4e-93f7-494d63e7e7ae (cwd None) >Thread-21::DEBUG::2014-10-05 21:17:05,503::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-21::DEBUG::2014-10-05 21:17:05,503::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-10-05 21:17:05,508::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000587044 s, 7.0 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-05 21:17:05,512::domainMonitor::264::Storage.DomainMonitorThread::(_monitorDomain) Domain ff559f46-c495-4f6b-901c-2a624042a050 changed its status to Valid >Thread-22::DEBUG::2014-10-05 21:17:05,512::misc::821::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Emitting event >Thread-21::INFO::2014-10-05 21:17:05,513::clusterlock::222::Storage.SANLock::(acquireHostId) Acquiring host id for domain ff559f46-c495-4f6b-901c-2a624042a050 (id: 3) >Thread-22::DEBUG::2014-10-05 21:17:05,513::misc::831::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `contEIOVms` >Thread-21::DEBUG::2014-10-05 21:17:05,514::clusterlock::240::Storage.SANLock::(acquireHostId) Host id for domain ff559f46-c495-4f6b-901c-2a624042a050 successfully acquired (id: 3) >Thread-22::DEBUG::2014-10-05 21:17:05,514::misc::841::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Event emitted >Thread-23::INFO::2014-10-05 21:17:05,520::clientIF::136::vds::(contEIOVms) vmContainerLock acquired >Thread-20::DEBUG::2014-10-05 21:17:05,638::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-20::DEBUG::2014-10-05 21:17:05,638::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-20::DEBUG::2014-10-05 21:17:05,638::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata count=1 (cwd None) >Thread-20::DEBUG::2014-10-05 21:17:05,643::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000458509 s, 4.5 MB/s\n'; <rc> = 0 >Thread-20::DEBUG::2014-10-05 21:17:05,643::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000458509 s, 4.5 MB/s'], size: 2048 >Thread-20::DEBUG::2014-10-05 21:17:05,643::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-20::DEBUG::2014-10-05 21:17:05,643::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-20::DEBUG::2014-10-05 21:17:05,644::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-20::DEBUG::2014-10-05 21:17:05,644::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm reload operation' is holding the operation mutex, waiting... >Thread-19::DEBUG::2014-10-05 21:17:05,647::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-19::DEBUG::2014-10-05 21:17:05,648::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-19::DEBUG::2014-10-05 21:17:05,648::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/dom_md >Thread-19::WARNING::2014-10-05 21:17:05,649::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/dom_md already exists >Thread-20::DEBUG::2014-10-05 21:17:05,649::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-19::DEBUG::2014-10-05 21:17:05,649::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/images >Thread-19::WARNING::2014-10-05 21:17:05,650::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/images already exists >Thread-20::DEBUG::2014-10-05 21:17:05,650::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-19::INFO::2014-10-05 21:17:05,651::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae to /rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae >Thread-20::DEBUG::2014-10-05 21:17:05,652::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-20::DEBUG::2014-10-05 21:17:05,652::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-19::DEBUG::2014-10-05 21:17:05,652::sp::1137::Storage.StoragePool::(_linkStorageDomain) Creating symlink from /rhev/data-center/mnt/blockSD/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae to /rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae >Thread-20::DEBUG::2014-10-05 21:17:05,653::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-19::DEBUG::2014-10-05 21:17:05,653::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050/images >Thread-19::WARNING::2014-10-05 21:17:05,654::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050/images already exists >Thread-20::DEBUG::2014-10-05 21:17:05,654::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060001|/dev/mapper/1IET_00060002|/dev/mapper/1IET_00060003|/dev/mapper/1IET_00060004|/dev/mapper/1IET_00060005|/dev/mapper/1IET_00060006|/dev/mapper/1IET_00060007|/dev/mapper/1IET_00060008|/dev/mapper/1IET_00060009|/dev/mapper/1IET_0006000a|/dev/mapper/3600144f09dbd05000000522dde040011|/dev/mapper/3600144f09dbd05000000522dde050012|/dev/mapper/3600144f09dbd05000000522dde070013|/dev/mapper/3600144f09dbd05000000522dde090014|/dev/mapper/3600144f09dbd05000000522dde0a0015|/dev/mapper/3600144f09dbd05000000522dde0c0016|/dev/mapper/3600144f09dbd05000000522dde0d0017|/dev/mapper/3600144f09dbd05000000522dde0f0018|/dev/mapper/3600144f09dbd05000000522dde100019|/dev/mapper/3600144f09dbd05000000522dde12001a|/dev/mapper/3600144f09dbd0500000053a84cba0002|/dev/mapper/3600144f09dbd0500000053a84e730003|/dev/mapper/3600144f09dbd0500000053a84f4d0004|/dev/mapper/3600144f09dbd0500000053a8503b0005|/dev/mapper/3600144f09dbd0500000053a850500006|/dev/mapper/3600144f09dbd0500000053a8505b0007|/dev/mapper/3600144f09dbd0500000053a850640008|/dev/mapper/3600144f09dbd0500000053a8506c0009|/dev/mapper/3600144f09dbd0500000053a85076000a|/dev/mapper/3600144f09dbd0500000053a8507f000b|/dev/mapper/3600144f09dbd0500000053a853ed000c|/dev/mapper/3600144f09dbd0500000053a85401000d|/dev/mapper/3600144f09dbd0500000053a85409000e|/dev/mapper/3600144f09dbd0500000053a85410000f|/dev/mapper/3600144f09dbd0500000053a854180010|/dev/mapper/3600144f09dbd0500000053a854250011|/dev/mapper/3600144f09dbd0500000053a8542d0012|/dev/mapper/3600144f09dbd0500000053a8543e0013|/dev/mapper/3600144f09dbd0500000053a854450014|/dev/mapper/3600144f09dbd0500000053a8544d0015|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name c73a3ced-4b41-4b4e-93f7-494d63e7e7ae (cwd None) >Thread-19::DEBUG::2014-10-05 21:17:05,655::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050/dom_md >Thread-19::WARNING::2014-10-05 21:17:05,659::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050/dom_md already exists >Thread-19::INFO::2014-10-05 21:17:05,660::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050 to /rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/ff559f46-c495-4f6b-901c-2a624042a050 >Thread-19::DEBUG::2014-10-05 21:17:05,660::sp::1137::Storage.StoragePool::(_linkStorageDomain) Creating symlink from /rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050 to /rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/ff559f46-c495-4f6b-901c-2a624042a050 >Thread-19::INFO::2014-10-05 21:17:05,660::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050 to /rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/mastersd >Thread-19::DEBUG::2014-10-05 21:17:05,660::sp::1137::Storage.StoragePool::(_linkStorageDomain) Creating symlink from /rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050 to /rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/mastersd >Thread-19::DEBUG::2014-10-05 21:17:05,660::threadPool::35::Storage.ThreadPool::(__init__) Enter - numThreads: 5, waitTimeout: 3, maxTasks: 500 >Thread-19::DEBUG::2014-10-05 21:17:05,662::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /bin/dd if=/rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/mastersd/dom_md/outbox iflag=direct,fullblock bs=512 count=8 skip=24 (cwd None) >Thread-19::DEBUG::2014-10-05 21:17:05,670::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.00396142 s, 1.0 MB/s\n'; <rc> = 0 >Thread-19::INFO::2014-10-05 21:17:05,671::storage_mailbox::385::Storage.MailBox.HsmMailMonitor::(_sendMail) HSM_MailMonitor sending mail to SPM - ['/bin/dd', 'of=/rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/mastersd/dom_md/inbox', 'iflag=fullblock', 'oflag=direct', 'conv=notrunc', 'bs=512', 'seek=24'] >Thread-19::DEBUG::2014-10-05 21:17:05,671::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /bin/dd of=/rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/mastersd/dom_md/inbox iflag=fullblock oflag=direct conv=notrunc bs=512 seek=24 (cwd None) >Thread-19::DEBUG::2014-10-05 21:17:05,683::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.00696694 s, 588 kB/s\n'; <rc> = 0 >Thread-19::DEBUG::2014-10-05 21:17:05,684::storage_mailbox::207::Storage.Mailbox.HSM::(__init__) HSM_MailboxMonitor created for pool b86b687a-d073-497a-ac8a-249025419a3e >Thread-19::DEBUG::2014-10-05 21:17:05,684::sp::462::Storage.StoragePool::(__createMailboxMonitor) HSM mailbox ready for pool b86b687a-d073-497a-ac8a-249025419a3e on master domain ff559f46-c495-4f6b-901c-2a624042a050 >Thread-19::DEBUG::2014-10-05 21:17:05,684::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.b86b687a-d073-497a-ac8a-249025419a3e' >Thread-19::DEBUG::2014-10-05 21:17:05,684::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.b86b687a-d073-497a-ac8a-249025419a3e' (0 active users) >Thread-19::DEBUG::2014-10-05 21:17:05,684::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.b86b687a-d073-497a-ac8a-249025419a3e' is free, finding out if anyone is waiting for it. >Thread-19::DEBUG::2014-10-05 21:17:05,685::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.b86b687a-d073-497a-ac8a-249025419a3e', Clearing records. >Thread-19::DEBUG::2014-10-05 21:17:05,685::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.HsmDomainMonitorLock' >Thread-19::DEBUG::2014-10-05 21:17:05,685::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.HsmDomainMonitorLock' (0 active users) >Thread-19::DEBUG::2014-10-05 21:17:05,685::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.HsmDomainMonitorLock' is free, finding out if anyone is waiting for it. >Thread-19::DEBUG::2014-10-05 21:17:05,685::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.HsmDomainMonitorLock', Clearing records. >Thread-19::INFO::2014-10-05 21:17:05,685::logUtils::47::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True >Thread-19::DEBUG::2014-10-05 21:17:05,686::task::1191::Storage.TaskManager.Task::(prepare) Task=`61433763-7cf9-4052-804b-79d9ee63238e`::finished: True >Thread-19::DEBUG::2014-10-05 21:17:05,686::task::595::Storage.TaskManager.Task::(_updateState) Task=`61433763-7cf9-4052-804b-79d9ee63238e`::moving from state preparing -> state finished >Thread-19::DEBUG::2014-10-05 21:17:05,686::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-19::DEBUG::2014-10-05 21:17:05,686::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-19::DEBUG::2014-10-05 21:17:05,686::task::993::Storage.TaskManager.Task::(_decref) Task=`61433763-7cf9-4052-804b-79d9ee63238e`::ref 0 aborting False >Thread-19::DEBUG::2014-10-05 21:17:05,686::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.connect' in bridge with True >Thread-19::DEBUG::2014-10-05 21:17:05,687::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:17:05,775::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:17:05,776::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-30::DEBUG::2014-10-05 21:17:05,777::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e95c212-9bce-4a58-8299-b746bf4423e6`::moving from state init -> state preparing >Thread-30::INFO::2014-10-05 21:17:05,777::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-30::INFO::2014-10-05 21:17:05,778::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': -1, 'acquired': False, 'delay': '0', 'lastCheck': '0.4', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000587044', 'lastCheck': '0.3', 'valid': True}} >Thread-30::DEBUG::2014-10-05 21:17:05,778::task::1191::Storage.TaskManager.Task::(prepare) Task=`5e95c212-9bce-4a58-8299-b746bf4423e6`::finished: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': -1, 'acquired': False, 'delay': '0', 'lastCheck': '0.4', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000587044', 'lastCheck': '0.3', 'valid': True}} >Thread-30::DEBUG::2014-10-05 21:17:05,778::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e95c212-9bce-4a58-8299-b746bf4423e6`::moving from state preparing -> state finished >Thread-30::DEBUG::2014-10-05 21:17:05,778::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-30::DEBUG::2014-10-05 21:17:05,778::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-30::DEBUG::2014-10-05 21:17:05,778::task::993::Storage.TaskManager.Task::(_decref) Task=`5e95c212-9bce-4a58-8299-b746bf4423e6`::ref 0 aborting False >Thread-30::DEBUG::2014-10-05 21:17:05,780::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:17:05,874::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:17:05,875::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-31::DEBUG::2014-10-05 21:17:05,875::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.setMOMPolicyParameters' in bridge with {u'key_value_store': {u'balloonEnabled': True, u'ksmEnabled': True}} >Thread-20::DEBUG::2014-10-05 21:17:05,891::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-20::DEBUG::2014-10-05 21:17:05,892::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-20::DEBUG::2014-10-05 21:17:05,893::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', u'PV1=pv:3600144f09dbd0500000053a84e730003,uuid:5zWyZx-cpAZ-tMON-3wa5-s6pN-Xj7N-b3fPs5,pestart:0,pecount:77,mapoffset:77', 'DESCRIPTION=sasha-el7', 'VERSION=3', u'PV2=pv:3600144f09dbd0500000053a853ed000c,uuid:c2LMVb-PXdj-XpTE-4f06-2rrh-TdAG-zzehcn,pestart:0,pecount:77,mapoffset:154', 'SDUUID=c73a3ced-4b41-4b4e-93f7-494d63e7e7ae', 'LOGBLKSIZE=512', 'TYPE=ISCSI', u'PV3=pv:3600144f09dbd0500000053a85401000d,uuid:1kMage-eAId-qy7G-oIqH-tfbH-ScLD-gqv7JM,pestart:0,pecount:77,mapoffset:231', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'PHYBLKSIZE=512', 'VGUUID=pF71zc-KPBX-BMKm-7KHK-KihY-ysp4-6DZTpA', 'CLASS=Data', 'LOCKPOLICY=', 'ROLE=Regular', u'PV0=pv:3600144f09dbd0500000053a85076000a,uuid:Ka3ewV-ddPQ-boL6-sDJT-xcFb-YBkc-9HeKFq,pestart:0,pecount:77,mapoffset:0', 'POOL_UUID=b86b687a-d073-497a-ac8a-249025419a3e', '_SHA_CKSUM=40993341560adfe664d3c1ab0c6b62068addc74c'] >Thread-20::DEBUG::2014-10-05 21:17:05,897::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae_imageNS' >Thread-20::DEBUG::2014-10-05 21:17:05,898::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae_volumeNS' >Thread-20::DEBUG::2014-10-05 21:17:05,899::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae_lvmActivationNS' >Thread-20::DEBUG::2014-10-05 21:17:05,901::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgck --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/3600144f09dbd0500000053a84e730003|/dev/mapper/3600144f09dbd0500000053a85076000a|/dev/mapper/3600144f09dbd0500000053a853ed000c|/dev/mapper/3600144f09dbd0500000053a85401000d|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' c73a3ced-4b41-4b4e-93f7-494d63e7e7ae (cwd None) >Thread-31::DEBUG::2014-10-05 21:17:05,936::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.setMOMPolicyParameters' in bridge with True >Thread-31::DEBUG::2014-10-05 21:17:05,936::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-20::DEBUG::2014-10-05 21:17:05,997::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-20::DEBUG::2014-10-05 21:17:05,997::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-20::DEBUG::2014-10-05 21:17:06,003::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000479383 s, 8.5 MB/s\n'; <rc> = 0 >Thread-20::DEBUG::2014-10-05 21:17:06,004::domainMonitor::264::Storage.DomainMonitorThread::(_monitorDomain) Domain c73a3ced-4b41-4b4e-93f7-494d63e7e7ae changed its status to Valid >Thread-32::DEBUG::2014-10-05 21:17:06,005::misc::821::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Emitting event >Thread-20::INFO::2014-10-05 21:17:06,005::clusterlock::222::Storage.SANLock::(acquireHostId) Acquiring host id for domain c73a3ced-4b41-4b4e-93f7-494d63e7e7ae (id: 3) >Thread-32::DEBUG::2014-10-05 21:17:06,006::misc::831::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `contEIOVms` >Thread-20::DEBUG::2014-10-05 21:17:06,007::clusterlock::240::Storage.SANLock::(acquireHostId) Host id for domain c73a3ced-4b41-4b4e-93f7-494d63e7e7ae successfully acquired (id: 3) >Thread-32::DEBUG::2014-10-05 21:17:06,007::misc::841::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Event emitted >Thread-33::INFO::2014-10-05 21:17:06,009::clientIF::136::vds::(contEIOVms) vmContainerLock acquired >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:17:09,093::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:17:09,094::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-34::DEBUG::2014-10-05 21:17:09,095::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:17:12,183::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:17:12,184::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-35::DEBUG::2014-10-05 21:17:12,185::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:17:15,273::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:17:15,279::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-36::DEBUG::2014-10-05 21:17:15,281::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-21::DEBUG::2014-10-05 21:17:15,515::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-10-05 21:17:15,522::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000703069 s, 5.8 MB/s\n'; <rc> = 0 >Thread-20::DEBUG::2014-10-05 21:17:16,008::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-20::DEBUG::2014-10-05 21:17:16,013::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000681433 s, 6.0 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:17:18,414::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:17:18,415::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-37::DEBUG::2014-10-05 21:17:18,417::task::595::Storage.TaskManager.Task::(_updateState) Task=`f5060e9c-b9ed-45f0-ace9-92c1b022b3c1`::moving from state init -> state preparing >Thread-37::INFO::2014-10-05 21:17:18,417::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-37::INFO::2014-10-05 21:17:18,417::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000681433', 'lastCheck': '2.4', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000703069', 'lastCheck': '2.9', 'valid': True}} >Thread-37::DEBUG::2014-10-05 21:17:18,417::task::1191::Storage.TaskManager.Task::(prepare) Task=`f5060e9c-b9ed-45f0-ace9-92c1b022b3c1`::finished: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000681433', 'lastCheck': '2.4', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000703069', 'lastCheck': '2.9', 'valid': True}} >Thread-37::DEBUG::2014-10-05 21:17:18,417::task::595::Storage.TaskManager.Task::(_updateState) Task=`f5060e9c-b9ed-45f0-ace9-92c1b022b3c1`::moving from state preparing -> state finished >Thread-37::DEBUG::2014-10-05 21:17:18,417::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-37::DEBUG::2014-10-05 21:17:18,418::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-37::DEBUG::2014-10-05 21:17:18,418::task::993::Storage.TaskManager.Task::(_decref) Task=`f5060e9c-b9ed-45f0-ace9-92c1b022b3c1`::ref 0 aborting False >Thread-37::DEBUG::2014-10-05 21:17:18,419::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:17:18,523::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:17:18,524::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-38::DEBUG::2014-10-05 21:17:18,525::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:17:21,666::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:17:21,666::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-39::DEBUG::2014-10-05 21:17:21,667::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:17:24,760::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:17:24,762::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-40::DEBUG::2014-10-05 21:17:24,762::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-21::DEBUG::2014-10-05 21:17:25,525::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-10-05 21:17:25,531::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000799463 s, 5.1 MB/s\n'; <rc> = 0 >Thread-20::DEBUG::2014-10-05 21:17:26,014::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-20::DEBUG::2014-10-05 21:17:26,019::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000609075 s, 6.7 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:17:27,850::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:17:27,851::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-41::DEBUG::2014-10-05 21:17:27,852::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:17:30,940::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:17:30,941::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-42::DEBUG::2014-10-05 21:17:30,942::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:17:34,031::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:17:34,032::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-43::DEBUG::2014-10-05 21:17:34,034::task::595::Storage.TaskManager.Task::(_updateState) Task=`5939ac4b-96f7-4039-a1ad-0163c9133f31`::moving from state init -> state preparing >Thread-43::INFO::2014-10-05 21:17:34,034::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-43::INFO::2014-10-05 21:17:34,034::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000609075', 'lastCheck': '8.0', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000799463', 'lastCheck': '8.5', 'valid': True}} >Thread-43::DEBUG::2014-10-05 21:17:34,034::task::1191::Storage.TaskManager.Task::(prepare) Task=`5939ac4b-96f7-4039-a1ad-0163c9133f31`::finished: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000609075', 'lastCheck': '8.0', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000799463', 'lastCheck': '8.5', 'valid': True}} >Thread-43::DEBUG::2014-10-05 21:17:34,034::task::595::Storage.TaskManager.Task::(_updateState) Task=`5939ac4b-96f7-4039-a1ad-0163c9133f31`::moving from state preparing -> state finished >Thread-43::DEBUG::2014-10-05 21:17:34,034::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-43::DEBUG::2014-10-05 21:17:34,034::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-43::DEBUG::2014-10-05 21:17:34,034::task::993::Storage.TaskManager.Task::(_decref) Task=`5939ac4b-96f7-4039-a1ad-0163c9133f31`::ref 0 aborting False >Thread-43::DEBUG::2014-10-05 21:17:34,036::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:17:34,140::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:17:34,141::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-44::DEBUG::2014-10-05 21:17:34,142::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-21::DEBUG::2014-10-05 21:17:35,536::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-10-05 21:17:35,541::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000721218 s, 5.7 MB/s\n'; <rc> = 0 >Thread-20::DEBUG::2014-10-05 21:17:36,021::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-20::DEBUG::2014-10-05 21:17:36,026::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000658657 s, 6.2 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:17:37,400::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:17:37,401::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-45::DEBUG::2014-10-05 21:17:37,402::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:17:40,492::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:17:40,493::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-46::DEBUG::2014-10-05 21:17:40,494::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:17:43,592::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:17:43,593::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-47::DEBUG::2014-10-05 21:17:43,595::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-21::DEBUG::2014-10-05 21:17:45,543::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-10-05 21:17:45,548::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000861392 s, 4.8 MB/s\n'; <rc> = 0 >Thread-20::DEBUG::2014-10-05 21:17:46,027::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-20::DEBUG::2014-10-05 21:17:46,032::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000630467 s, 6.5 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:17:46,685::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:17:46,686::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-48::DEBUG::2014-10-05 21:17:46,687::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >MainThread::DEBUG::2014-10-05 21:17:47,145::vdsm::58::vds::(sigtermHandler) Received signal 15 >MainThread::DEBUG::2014-10-05 21:17:47,146::protocoldetector::135::vds.MultiProtocolAcceptor::(stop) Stopping Acceptor >MainThread::INFO::2014-10-05 21:17:47,148::__init__::563::jsonrpc.JsonRpcServer::(stop) Stopping JsonRPC Server >Detector thread::DEBUG::2014-10-05 21:17:47,148::protocoldetector::106::vds.MultiProtocolAcceptor::(_cleanup) Cleaning Acceptor >MainThread::INFO::2014-10-05 21:17:47,149::vmchannels::188::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2014-10-05 21:17:47,149::momIF::91::MOM::(stop) Shutting down MOM >MainThread::DEBUG::2014-10-05 21:17:47,150::task::595::Storage.TaskManager.Task::(_updateState) Task=`cf3f565b-9813-4426-abbf-006b5e52c591`::moving from state init -> state preparing >MainThread::INFO::2014-10-05 21:17:47,150::logUtils::44::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2014-10-05 21:17:47,151::storageServer::706::Storage.ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2014-10-05 21:17:47,156::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/04511d19-f03a-469a-9e31-ab51d6e75a56/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:17:47,156::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/67ba7199-44ec-4562-9c17-ebc3903c7c7f/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:17:47,157::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/83d26f74-2a29-4025-8210-5ab61cb49a82/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:17:47,157::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/fd98c9a9-23f9-43f5-8e7b-4022f5bdd683/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:17:47,157::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/b9c2b835-cdbd-48ea-8d69-7a33042c79b1/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:17:47,158::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/c175b488-c93a-4dff-b709-b137853eab67/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:17:47,158::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/6ca7f5aa-6229-4549-9c6f-54ec7b7099a6/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:17:47,159::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/681bd5bc-4d98-4da3-bd9a-cd37eec7b9af/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:17:47,159::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050/master` is not mounted, skipping >VM Channels Listener::INFO::2014-10-05 21:17:47,464::vmchannels::183::vds::(run) VM channels listener thread has ended. >MainThread::INFO::2014-10-05 21:17:47,664::domainMonitor::137::Storage.DomainMonitor::(close) Stopping domain monitors >MainThread::INFO::2014-10-05 21:17:47,665::domainMonitor::114::Storage.DomainMonitor::(stopMonitoring) Stop monitoring c73a3ced-4b41-4b4e-93f7-494d63e7e7ae >Thread-20::DEBUG::2014-10-05 21:17:47,665::domainMonitor::192::Storage.DomainMonitorThread::(_monitorLoop) Stopping domain monitor for c73a3ced-4b41-4b4e-93f7-494d63e7e7ae >Thread-20::INFO::2014-10-05 21:17:47,665::clusterlock::245::Storage.SANLock::(releaseHostId) Releasing host id for domain c73a3ced-4b41-4b4e-93f7-494d63e7e7ae (id: 3) >Thread-29::INFO::2014-10-05 21:17:47,687::storage_mailbox::509::Storage.MailBox.HsmMailMonitor::(run) HSM_MailboxMonitor - Incoming mail monitoring thread stopped, clearing outgoing mail >Thread-29::INFO::2014-10-05 21:17:47,688::storage_mailbox::385::Storage.MailBox.HsmMailMonitor::(_sendMail) HSM_MailMonitor sending mail to SPM - ['/bin/dd', 'of=/rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/mastersd/dom_md/inbox', 'iflag=fullblock', 'oflag=direct', 'conv=notrunc', 'bs=512', 'seek=24'] >Thread-29::DEBUG::2014-10-05 21:17:47,688::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /bin/dd of=/rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/mastersd/dom_md/inbox iflag=fullblock oflag=direct conv=notrunc bs=512 seek=24 (cwd None) >Thread-29::DEBUG::2014-10-05 21:17:47,694::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.00320464 s, 1.3 MB/s\n'; <rc> = 0 >Thread-20::DEBUG::2014-10-05 21:17:49,666::clusterlock::255::Storage.SANLock::(releaseHostId) Host id for domain c73a3ced-4b41-4b4e-93f7-494d63e7e7ae released successfully (id: 3) >MainThread::INFO::2014-10-05 21:17:49,666::domainMonitor::114::Storage.DomainMonitor::(stopMonitoring) Stop monitoring ff559f46-c495-4f6b-901c-2a624042a050 >Thread-21::DEBUG::2014-10-05 21:17:49,667::domainMonitor::192::Storage.DomainMonitorThread::(_monitorLoop) Stopping domain monitor for ff559f46-c495-4f6b-901c-2a624042a050 >Thread-21::INFO::2014-10-05 21:17:49,667::clusterlock::245::Storage.SANLock::(releaseHostId) Releasing host id for domain ff559f46-c495-4f6b-901c-2a624042a050 (id: 3) >Thread-21::DEBUG::2014-10-05 21:17:51,667::clusterlock::255::Storage.SANLock::(releaseHostId) Host id for domain ff559f46-c495-4f6b-901c-2a624042a050 released successfully (id: 3) >MainThread::DEBUG::2014-10-05 21:17:51,668::taskManager::90::Storage.TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2014-10-05 21:17:51,668::logUtils::47::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2014-10-05 21:17:51,668::task::1191::Storage.TaskManager.Task::(prepare) Task=`cf3f565b-9813-4426-abbf-006b5e52c591`::finished: None >MainThread::DEBUG::2014-10-05 21:17:51,668::task::595::Storage.TaskManager.Task::(_updateState) Task=`cf3f565b-9813-4426-abbf-006b5e52c591`::moving from state preparing -> state finished >MainThread::DEBUG::2014-10-05 21:17:51,668::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2014-10-05 21:17:51,668::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2014-10-05 21:17:51,668::task::993::Storage.TaskManager.Task::(_decref) Task=`cf3f565b-9813-4426-abbf-006b5e52c591`::ref 0 aborting False >MainThread::INFO::2014-10-05 21:17:51,668::vdsm::137::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2014-10-05 21:17:51,668::vdsm::141::vds::(run) <_MainThread(MainThread, started 140653274130176)> >MainThread::INFO::2014-10-05 21:17:51,668::vdsm::141::vds::(run) <Thread(libvirtEventLoop, started daemon 140653185271552)> >MainThread::INFO::2014-10-05 21:17:51,669::vdsm::141::vds::(run) <WorkerThread(Thread-8, started daemon 140652765497088)> >MainThread::INFO::2014-10-05 21:17:51,669::vdsm::141::vds::(run) <WorkerThread(Thread-10, started daemon 140652744517376)> >MainThread::INFO::2014-10-05 21:17:51,669::vdsm::141::vds::(run) <WorkerThread(Thread-6, started daemon 140652786476800)> >MainThread::INFO::2014-10-05 21:17:51,669::vdsm::141::vds::(run) <WorkerThread(Thread-9, started daemon 140652755007232)> >MainThread::INFO::2014-10-05 21:17:51,669::momIF::91::MOM::(stop) Shutting down MOM >MainThread::INFO::2014-10-05 21:17:51,669::vdsm::141::vds::(run) <MomThread(MOM, started 140652182497024)> >MainThread::INFO::2014-10-05 21:17:51,669::vdsm::141::vds::(run) <WorkerThread(Thread-5, started daemon 140652796966656)> >MainThread::INFO::2014-10-05 21:17:51,669::vdsm::141::vds::(run) <WorkerThread(Thread-7, started daemon 140652775986944)> >MainThread::INFO::2014-10-05 21:17:51,669::vdsm::141::vds::(run) <PolicyEngine(PolicyEngine, started daemon 140651681265408)> >MainThread::INFO::2014-10-05 21:17:51,669::vdsm::141::vds::(run) <WorkerThread(Thread-4, started daemon 140653143312128)> >MainThread::INFO::2014-10-05 21:17:51,669::vdsm::141::vds::(run) <WorkerThread(Thread-3, started daemon 140653153801984)> >MainThread::INFO::2014-10-05 21:17:51,669::vdsm::141::vds::(run) <WorkerThread(Thread-2, started daemon 140653164291840)> >MainThread::INFO::2014-10-05 21:17:51,669::vdsm::141::vds::(run) <WorkerThread(Thread-1, started daemon 140653174781696)> >MainThread::INFO::2014-10-05 21:18:37,910::vdsm::131::vds::(run) (PID: 3057) I am the actual vdsm 4.16.6-0.el6 voodoo1.tlv.redhat.com (2.6.32-431.29.2.el6.x86_64) >MainThread::DEBUG::2014-10-05 21:18:37,920::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2014-10-05 21:18:37,921::threadPool::35::Storage.ThreadPool::(__init__) Enter - numThreads: 10, waitTimeout: 3, maxTasks: 500 >MainThread::DEBUG::2014-10-05 21:18:37,922::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt >MainThread::WARNING::2014-10-05 21:18:37,923::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt already exists >MainThread::DEBUG::2014-10-05 21:18:37,929::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/04511d19-f03a-469a-9e31-ab51d6e75a56/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:18:37,931::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/67ba7199-44ec-4562-9c17-ebc3903c7c7f/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:18:37,932::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/83d26f74-2a29-4025-8210-5ab61cb49a82/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:18:37,932::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/fd98c9a9-23f9-43f5-8e7b-4022f5bdd683/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:18:37,932::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/b9c2b835-cdbd-48ea-8d69-7a33042c79b1/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:18:37,933::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/c175b488-c93a-4dff-b709-b137853eab67/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:18:37,933::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/6ca7f5aa-6229-4549-9c6f-54ec7b7099a6/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:18:37,934::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/681bd5bc-4d98-4da3-bd9a-cd37eec7b9af/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:18:37,934::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:18:37,987::supervdsm::77::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2014-10-05 21:18:37,999::multipath::142::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2014-10-05 21:18:37,999::hsm::418::Storage.Misc.excCmd::(__validateLvmLockingType) /usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type (cwd None) >MainThread::DEBUG::2014-10-05 21:18:38,099::hsm::418::Storage.Misc.excCmd::(__validateLvmLockingType) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2014-10-05 21:18:38,099::hsm::442::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >MainThread::DEBUG::2014-10-05 21:18:38,101::hsm::474::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >MainThread::DEBUG::2014-10-05 21:18:38,101::hsm::475::Storage.HSM::(__cleanStorageRepository) Mount list: [] >MainThread::DEBUG::2014-10-05 21:18:38,101::hsm::477::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >MainThread::DEBUG::2014-10-05 21:18:38,104::hsm::520::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >storageRefresh::DEBUG::2014-10-05 21:18:38,104::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2014-10-05 21:18:38,105::dispatcher::46::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >storageRefresh::DEBUG::2014-10-05 21:18:38,106::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >storageRefresh::DEBUG::2014-10-05 21:18:38,119::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >storageRefresh::DEBUG::2014-10-05 21:18:38,120::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >storageRefresh::DEBUG::2014-10-05 21:18:38,121::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >storageRefresh::DEBUG::2014-10-05 21:18:38,123::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >storageRefresh::DEBUG::2014-10-05 21:18:38,155::misc::751::Storage.SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2014-10-05 21:18:38,166::task::595::Storage.TaskManager.Task::(_updateState) Task=`c40e3395-48d9-43dc-ae58-792ec033f691`::moving from state init -> state preparing >MainThread::INFO::2014-10-05 21:18:38,166::logUtils::44::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback(callbackFunc=<bound method clientIF.contEIOVms of <clientIF.clientIF object at 0x31e24d0>>) >MainThread::INFO::2014-10-05 21:18:38,166::logUtils::47::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback, Return response: None >MainThread::DEBUG::2014-10-05 21:18:38,167::task::1191::Storage.TaskManager.Task::(prepare) Task=`c40e3395-48d9-43dc-ae58-792ec033f691`::finished: None >MainThread::DEBUG::2014-10-05 21:18:38,167::task::595::Storage.TaskManager.Task::(_updateState) Task=`c40e3395-48d9-43dc-ae58-792ec033f691`::moving from state preparing -> state finished >MainThread::DEBUG::2014-10-05 21:18:38,167::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2014-10-05 21:18:38,167::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2014-10-05 21:18:38,167::task::993::Storage.TaskManager.Task::(_decref) Task=`c40e3395-48d9-43dc-ae58-792ec033f691`::ref 0 aborting False >storageRefresh::DEBUG::2014-10-05 21:18:38,167::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >MainThread::INFO::2014-10-05 21:18:38,173::momIF::49::MOM::(__init__) Starting up MOM >storageRefresh::DEBUG::2014-10-05 21:18:38,177::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >storageRefresh::DEBUG::2014-10-05 21:18:38,179::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >MainThread::INFO::2014-10-05 21:18:38,180::vmchannels::192::vds::(settimeout) Setting channels' timeout to 30 seconds. >storageRefresh::DEBUG::2014-10-05 21:18:38,180::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >VM Channels Listener::INFO::2014-10-05 21:18:38,182::vmchannels::175::vds::(run) Starting VM channels listener thread. >storageRefresh::DEBUG::2014-10-05 21:18:38,182::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >clientIFinit::DEBUG::2014-10-05 21:18:38,183::libvirtconnection::150::root::(get) trying to connect libvirt >storageRefresh::DEBUG::2014-10-05 21:18:38,183::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-10-05 21:18:38,183::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >storageRefresh::DEBUG::2014-10-05 21:18:38,184::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-10-05 21:18:38,184::misc::751::Storage.SamplingMethod::(__call__) Returning last result >storageRefresh::DEBUG::2014-10-05 21:18:38,184::lvm::317::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >storageRefresh::DEBUG::2014-10-05 21:18:38,186::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm pvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size (cwd None) >MainThread::DEBUG::2014-10-05 21:18:38,244::protocoldetector::131::vds.MultiProtocolAcceptor::(add_detector) adding detector: <rpc.BindingXMLRPC.XmlDetector instance at 0x32da830> >storageRefresh::DEBUG::2014-10-05 21:18:38,273::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >storageRefresh::DEBUG::2014-10-05 21:18:38,276::lvm::342::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >storageRefresh::DEBUG::2014-10-05 21:18:38,282::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >storageRefresh::DEBUG::2014-10-05 21:18:38,283::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >MainThread::DEBUG::2014-10-05 21:18:38,303::protocoldetector::131::vds.MultiProtocolAcceptor::(add_detector) adding detector: <yajsonrpc.stompReactor.StompDetector instance at 0x34a65a8> >JsonRpcServer::DEBUG::2014-10-05 21:18:38,304::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Detector thread::DEBUG::2014-10-05 21:18:38,306::protocoldetector::70::vds.MultiProtocolAcceptor::(serve_forever) Acceptor running >Detector thread::DEBUG::2014-10-05 21:18:38,306::protocoldetector::72::vds.MultiProtocolAcceptor::(serve_forever) Using required_size=7 >storageRefresh::DEBUG::2014-10-05 21:18:38,403::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-10-05 21:18:38,403::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >storageRefresh::DEBUG::2014-10-05 21:18:38,404::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags (cwd None) >storageRefresh::DEBUG::2014-10-05 21:18:38,475::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-10-05 21:18:38,476::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >storageRefresh::DEBUG::2014-10-05 21:18:38,476::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >storageRefresh::DEBUG::2014-10-05 21:18:38,545::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-10-05 21:18:38,545::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >storageRefresh::DEBUG::2014-10-05 21:18:38,545::hsm::387::Storage.HSM::(storageRefresh) HSM is ready >Detector thread::DEBUG::2014-10-05 21:20:50,792::protocoldetector::166::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.201.227:58506 >Detector thread::DEBUG::2014-10-05 21:20:50,872::protocoldetector::177::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.201.227:58506 >Detector thread::DEBUG::2014-10-05 21:20:50,872::protocoldetector::203::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol stomp from 10.35.201.227:58506 >Detector thread::DEBUG::2014-10-05 21:20:50,872::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='CONNECT'> >Detector thread::INFO::2014-10-05 21:20:50,872::stompReactor::66::Broker.StompAdapter::(_cmd_connect) Processing CONNECT request >Detector thread::DEBUG::2014-10-05 21:20:50,873::stompReactor::338::protocoldetector.StompDetector::(handleSocket) Stomp detected from ('10.35.201.227', 58506) >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:20:50,953::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SUBSCRIBE'> >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:20:50,953::stompReactor::89::Broker.StompAdapter::(_cmd_subscribe) Subscribe command ignored >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:20:51,073::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:20:51,074::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-13::DEBUG::2014-10-05 21:20:51,074::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getCapabilities' in bridge with {} >Thread-13::DEBUG::2014-10-05 21:20:51,110::utils::738::root::(execCmd) /sbin/ip route show to 0.0.0.0/0 table all (cwd None) >Thread-13::DEBUG::2014-10-05 21:20:51,116::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:20:51,127::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/bonding_masters (cwd None) >Thread-13::DEBUG::2014-10-05 21:20:51,138::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:20:51,138::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/KKFnyE8102iJRLc/bonding/mode (cwd None) >Thread-13::DEBUG::2014-10-05 21:20:51,146::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:20:51,147::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/KKFnyE8102iJRLc/bonding/mode (cwd None) >Thread-13::DEBUG::2014-10-05 21:20:51,154::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:20:51,155::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/KKFnyE8102iJRLc/bonding/mode (cwd None) >Thread-13::DEBUG::2014-10-05 21:20:51,163::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:20:51,164::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/KKFnyE8102iJRLc/bonding/mode (cwd None) >Thread-13::DEBUG::2014-10-05 21:20:51,171::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:20:51,172::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/KKFnyE8102iJRLc/bonding/mode (cwd None) >Thread-13::DEBUG::2014-10-05 21:20:51,180::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:20:51,182::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/KKFnyE8102iJRLc/bonding/mode (cwd None) >Thread-13::DEBUG::2014-10-05 21:20:51,189::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:20:51,190::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/KKFnyE8102iJRLc/bonding/mode (cwd None) >Thread-13::DEBUG::2014-10-05 21:20:51,197::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:20:51,198::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/bonding_masters (cwd None) >Thread-13::DEBUG::2014-10-05 21:20:51,230::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:20:51,282::caps::646::root::(get) VirtioRNG DISABLED: libvirt version 0.10.2-29.el6_5.12 required >= 0.10.2-31 >Thread-13::DEBUG::2014-10-05 21:20:51,282::utils::738::root::(execCmd) numactl --hardware (cwd None) >Thread-13::DEBUG::2014-10-05 21:20:51,286::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:20:51,286::utils::738::root::(execCmd) sysctl -n -e kernel.numa_balancing (cwd None) >Thread-13::DEBUG::2014-10-05 21:20:51,289::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:20:51,292::caps::269::root::(getLiveMergeSupport) libvirt is missing 'VIR_DOMAIN_BLOCK_COMMIT_RELATIVE': live merge disabled >Thread-13::DEBUG::2014-10-05 21:20:51,293::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.getCapabilities' in bridge with {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:nsoffer1'}], 'FC': []}, 'packages2': {'kernel': {'release': '431.29.2.el6.x86_64', 'buildtime': 1406465746.0, 'version': '2.6.32'}, 'spice-server': {'release': '6.el6_5.2', 'buildtime': 1401730350L, 'version': '0.12.4'}, 'vdsm': {'release': '0.el6', 'buildtime': 1412232469L, 'version': '4.16.6'}, 'qemu-kvm': {'release': '2.415.el6_5.14', 'buildtime': 1408575896L, 'version': '0.12.1.2'}, 'libvirt': {'release': '29.el6_5.12', 'buildtime': 1408610144L, 'version': '0.10.2'}, 'qemu-img': {'release': '2.415.el6_5.14', 'buildtime': 1408575896L, 'version': '0.12.1.2'}, 'mom': {'release': '0.0.master.20140911.gitcf7ff17.el6', 'buildtime': 1410462635L, 'version': '0.4.2'}}, 'numaNodeDistance': {'0': [10]}, 'cpuModel': 'Intel Core Processor (Haswell)', 'liveMerge': 'false', 'hooks': {}, 'cpuSockets': '8', 'vmTypes': ['kvm'], 'selinux': {'mode': '1'}, 'kdumpStatus': 1, 'supportedProtocols': ['2.2', '2.3'], 'networks': {'ovirtmgmt': {'iface': 'ovirtmgmt', 'addr': '10.35.0.100', 'cfg': {'DEFROUTE': 'yes', 'HOTPLUG': 'no', 'MTU': '1500', 'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'STP': 'off', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'bridged': True, 'ipv6addrs': ['fe80::5054:ff:fe87:196f/64'], 'gateway': '10.35.1.254', 'bootproto4': 'dhcp', 'netmask': '255.255.254.0', 'stp': 'off', 'ipv4addrs': ['10.35.0.100/23'], 'mtu': '1500', 'ipv6gateway': '::', 'ports': ['eth0']}}, 'bridges': {'ovirtmgmt': {'addr': '10.35.0.100', 'cfg': {'DEFROUTE': 'yes', 'HOTPLUG': 'no', 'MTU': '1500', 'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'STP': 'off', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'ipv6addrs': ['fe80::5054:ff:fe87:196f/64'], 'mtu': '1500', 'netmask': '255.255.254.0', 'stp': 'off', 'ipv4addrs': ['10.35.0.100/23'], 'ipv6gateway': '::', 'gateway': '10.35.1.254', 'opts': {'topology_change_detected': '0', 'multicast_last_member_count': '2', 'hash_elasticity': '4', 'multicast_query_response_interval': '999', 'multicast_snooping': '1', 'multicast_startup_query_interval': '3124', 'hello_timer': '128', 'multicast_querier_interval': '25496', 'max_age': '1999', 'hash_max': '512', 'stp_state': '0', 'root_id': '8000.52540087196f', 'priority': '32768', 'multicast_membership_interval': '25996', 'root_path_cost': '0', 'root_port': '0', 'multicast_querier': '0', 'multicast_startup_query_count': '2', 'hello_time': '199', 'topology_change': '0', 'bridge_id': '8000.52540087196f', 'topology_change_timer': '0', 'ageing_time': '29995', 'gc_timer': '28', 'group_addr': '1:80:c2:0:0:0', 'tcn_timer': '0', 'multicast_query_interval': '12498', 'multicast_last_member_interval': '99', 'multicast_router': '1', 'forward_delay': '0'}, 'ports': ['eth0']}}, 'uuid': '85CFDED0-FD24-4645-AF7C-D47A088AE9E8', 'lastClientIface': 'ovirtmgmt', 'nics': {'eth0': {'addr': '', 'cfg': {'BRIDGE': 'ovirtmgmt', 'NM_CONTROLLED': 'no', 'MTU': '1500', 'HWADDR': '52:54:00:87:19:6f', 'DEVICE': 'eth0', 'ONBOOT': 'yes'}, 'ipv6addrs': [], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': '52:54:00:87:19:6f', 'speed': 0}}, 'software_revision': '0', 'clusterLevels': ['3.0', '3.1', '3.2', '3.3', '3.4', '3.5'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,mmx,fxsr,sse,sse2,ss,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,rep_good,unfair_spinlock,pni,pclmulqdq,vmx,ssse3,fma,cx16,pcid,sse4_1,sse4_2,x2apic,movbe,popcnt,tsc_deadline_timer,aes,xsave,avx,f16c,rdrand,hypervisor,lahf_lm,abm,xsaveopt,vnmi,ept,fsgsbase,bmi1,hle,avx2,smep,bmi2,erms,invpcid,rtm,model_Haswell,model_Nehalem,model_Conroe,model_Penryn,model_Westmere,model_SandyBridge', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:nsoffer1', 'netConfigDirty': 'False', 'supportedENGINEs': ['3.0', '3.1', '3.2', '3.3', '3.4', '3.5'], 'autoNumaBalancing': 2, 'reservedMem': '321', 'bondings': {'bond4': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond0': {'addr': '', 'cfg': {}, 'ipv6addrs': [], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': '00:00:00:00:00:00', 'slaves': [], 'opts': {}}, 'bond1': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond2': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond3': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}}, 'software_version': '4.16', 'memSize': '3734', 'cpuSpeed': '3392.160', 'numaNodes': {u'0': {'totalMemory': '3734', 'cpus': [0, 1, 2, 3, 4, 5, 6, 7]}}, 'version_name': 'Snow Man', 'vlans': {}, 'cpuCores': '8', 'kvmEnabled': 'true', 'guestOverhead': '65', 'cpuThreads': '8', 'emulatedMachines': [u'rhel6.5.0', u'pc', u'rhel6.4.0', u'rhel6.3.0', u'rhel6.2.0', u'rhel6.1.0', u'rhel6.0.0', u'rhel5.5.0', u'rhel5.4.4', u'rhel5.4.0'], 'operatingSystem': {'release': '6.5.0.1.el6', 'version': '6Server', 'name': 'RHEL'}} >Thread-13::DEBUG::2014-10-05 21:20:51,294::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:20:51,386::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:20:51,387::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-14::DEBUG::2014-10-05 21:20:51,387::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getHardwareInfo' in bridge with {} >Thread-14::DEBUG::2014-10-05 21:20:51,421::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.getHardwareInfo' in bridge with {'systemProductName': 'Bochs', 'systemUUID': '85cfded0-fd24-4645-af7c-d47a088ae9e8', 'systemManufacturer': 'Bochs'} >Thread-14::DEBUG::2014-10-05 21:20:51,422::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:20:53,437::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:20:53,438::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-15::DEBUG::2014-10-05 21:20:53,439::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:20:56,534::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:20:56,535::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-16::DEBUG::2014-10-05 21:20:56,536::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:20:59,652::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:20:59,653::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-17::DEBUG::2014-10-05 21:20:59,655::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:21:02,748::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:21:02,748::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-18::DEBUG::2014-10-05 21:21:02,749::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:21:05,842::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:21:05,843::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-19::DEBUG::2014-10-05 21:21:05,844::task::595::Storage.TaskManager.Task::(_updateState) Task=`fe9a535a-22b7-4f17-b4d8-e9758420d770`::moving from state init -> state preparing >Thread-19::INFO::2014-10-05 21:21:05,844::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-19::INFO::2014-10-05 21:21:05,844::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-19::DEBUG::2014-10-05 21:21:05,844::task::1191::Storage.TaskManager.Task::(prepare) Task=`fe9a535a-22b7-4f17-b4d8-e9758420d770`::finished: {} >Thread-19::DEBUG::2014-10-05 21:21:05,844::task::595::Storage.TaskManager.Task::(_updateState) Task=`fe9a535a-22b7-4f17-b4d8-e9758420d770`::moving from state preparing -> state finished >Thread-19::DEBUG::2014-10-05 21:21:05,845::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-19::DEBUG::2014-10-05 21:21:05,845::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-19::DEBUG::2014-10-05 21:21:05,845::task::993::Storage.TaskManager.Task::(_decref) Task=`fe9a535a-22b7-4f17-b4d8-e9758420d770`::ref 0 aborting False >Thread-19::DEBUG::2014-10-05 21:21:05,846::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:21:05,946::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:21:05,946::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-20::DEBUG::2014-10-05 21:21:05,947::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:21:09,125::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:21:09,126::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-21::DEBUG::2014-10-05 21:21:09,127::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:21:12,216::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:21:12,217::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-22::DEBUG::2014-10-05 21:21:12,217::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:21:15,306::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:21:15,307::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-23::DEBUG::2014-10-05 21:21:15,308::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:21:18,438::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:21:18,439::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-24::DEBUG::2014-10-05 21:21:18,439::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:21:21,528::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:21:21,529::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-25::DEBUG::2014-10-05 21:21:21,530::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b979a58-efb9-467a-b9e3-00f901cdd998`::moving from state init -> state preparing >Thread-25::INFO::2014-10-05 21:21:21,531::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-25::INFO::2014-10-05 21:21:21,531::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-25::DEBUG::2014-10-05 21:21:21,531::task::1191::Storage.TaskManager.Task::(prepare) Task=`1b979a58-efb9-467a-b9e3-00f901cdd998`::finished: {} >Thread-25::DEBUG::2014-10-05 21:21:21,531::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b979a58-efb9-467a-b9e3-00f901cdd998`::moving from state preparing -> state finished >Thread-25::DEBUG::2014-10-05 21:21:21,531::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-25::DEBUG::2014-10-05 21:21:21,531::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-25::DEBUG::2014-10-05 21:21:21,531::task::993::Storage.TaskManager.Task::(_decref) Task=`1b979a58-efb9-467a-b9e3-00f901cdd998`::ref 0 aborting False >Thread-25::DEBUG::2014-10-05 21:21:21,532::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:21:21,621::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:21:21,622::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-26::DEBUG::2014-10-05 21:21:21,623::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:21:24,778::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:21:24,779::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-27::DEBUG::2014-10-05 21:21:24,780::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:21:27,877::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:21:27,887::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-28::DEBUG::2014-10-05 21:21:27,889::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:21:30,979::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:21:30,980::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-29::DEBUG::2014-10-05 21:21:30,981::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:21:34,074::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:21:34,075::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-30::DEBUG::2014-10-05 21:21:34,077::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:21:37,164::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:21:37,165::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-31::DEBUG::2014-10-05 21:21:37,166::task::595::Storage.TaskManager.Task::(_updateState) Task=`f76a1f6e-63b6-40c7-8711-a11218f99ceb`::moving from state init -> state preparing >Thread-31::INFO::2014-10-05 21:21:37,166::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-31::INFO::2014-10-05 21:21:37,166::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-31::DEBUG::2014-10-05 21:21:37,166::task::1191::Storage.TaskManager.Task::(prepare) Task=`f76a1f6e-63b6-40c7-8711-a11218f99ceb`::finished: {} >Thread-31::DEBUG::2014-10-05 21:21:37,166::task::595::Storage.TaskManager.Task::(_updateState) Task=`f76a1f6e-63b6-40c7-8711-a11218f99ceb`::moving from state preparing -> state finished >Thread-31::DEBUG::2014-10-05 21:21:37,166::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-31::DEBUG::2014-10-05 21:21:37,166::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-31::DEBUG::2014-10-05 21:21:37,166::task::993::Storage.TaskManager.Task::(_decref) Task=`f76a1f6e-63b6-40c7-8711-a11218f99ceb`::ref 0 aborting False >Thread-31::DEBUG::2014-10-05 21:21:37,168::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:21:37,262::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:21:37,262::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-32::DEBUG::2014-10-05 21:21:37,263::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:21:40,431::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:21:40,432::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-33::DEBUG::2014-10-05 21:21:40,432::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:21:43,523::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:21:43,523::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-34::DEBUG::2014-10-05 21:21:43,524::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:21:46,647::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:21:46,648::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-35::DEBUG::2014-10-05 21:21:46,649::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:21:49,736::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:21:49,737::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-36::DEBUG::2014-10-05 21:21:49,738::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:21:52,824::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:21:52,825::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-37::DEBUG::2014-10-05 21:21:52,826::task::595::Storage.TaskManager.Task::(_updateState) Task=`50fa663a-a14d-4817-9c18-d9025f68e3c3`::moving from state init -> state preparing >Thread-37::INFO::2014-10-05 21:21:52,826::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-37::INFO::2014-10-05 21:21:52,826::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-37::DEBUG::2014-10-05 21:21:52,826::task::1191::Storage.TaskManager.Task::(prepare) Task=`50fa663a-a14d-4817-9c18-d9025f68e3c3`::finished: {} >Thread-37::DEBUG::2014-10-05 21:21:52,826::task::595::Storage.TaskManager.Task::(_updateState) Task=`50fa663a-a14d-4817-9c18-d9025f68e3c3`::moving from state preparing -> state finished >Thread-37::DEBUG::2014-10-05 21:21:52,826::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-37::DEBUG::2014-10-05 21:21:52,826::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-37::DEBUG::2014-10-05 21:21:52,826::task::993::Storage.TaskManager.Task::(_decref) Task=`50fa663a-a14d-4817-9c18-d9025f68e3c3`::ref 0 aborting False >Thread-37::DEBUG::2014-10-05 21:21:52,828::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:21:52,922::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:21:52,923::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-38::DEBUG::2014-10-05 21:21:52,924::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:21:56,240::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:21:56,241::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-39::DEBUG::2014-10-05 21:21:56,243::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:21:59,017::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:21:59,018::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-40::DEBUG::2014-10-05 21:21:59,019::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:22:02,112::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:22:02,113::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-41::DEBUG::2014-10-05 21:22:02,113::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:22:05,202::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:22:05,203::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-42::DEBUG::2014-10-05 21:22:05,205::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:22:08,292::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:22:08,293::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-43::DEBUG::2014-10-05 21:22:08,296::task::595::Storage.TaskManager.Task::(_updateState) Task=`c1ee63c9-fcac-4447-88cd-6a54d23fcbf6`::moving from state init -> state preparing >Thread-43::INFO::2014-10-05 21:22:08,297::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-43::INFO::2014-10-05 21:22:08,297::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-43::DEBUG::2014-10-05 21:22:08,298::task::1191::Storage.TaskManager.Task::(prepare) Task=`c1ee63c9-fcac-4447-88cd-6a54d23fcbf6`::finished: {} >Thread-43::DEBUG::2014-10-05 21:22:08,298::task::595::Storage.TaskManager.Task::(_updateState) Task=`c1ee63c9-fcac-4447-88cd-6a54d23fcbf6`::moving from state preparing -> state finished >Thread-43::DEBUG::2014-10-05 21:22:08,298::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-43::DEBUG::2014-10-05 21:22:08,298::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-43::DEBUG::2014-10-05 21:22:08,299::task::993::Storage.TaskManager.Task::(_decref) Task=`c1ee63c9-fcac-4447-88cd-6a54d23fcbf6`::ref 0 aborting False >Thread-43::DEBUG::2014-10-05 21:22:08,306::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:22:08,396::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:22:08,397::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-44::DEBUG::2014-10-05 21:22:08,398::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:22:11,552::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:22:11,553::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-45::DEBUG::2014-10-05 21:22:11,554::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:22:14,640::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:22:14,640::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-46::DEBUG::2014-10-05 21:22:14,642::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:22:17,731::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:22:17,732::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-47::DEBUG::2014-10-05 21:22:17,737::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:22:20,824::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:22:20,825::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-48::DEBUG::2014-10-05 21:22:20,826::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:22:23,916::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:22:23,917::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-49::DEBUG::2014-10-05 21:22:23,919::task::595::Storage.TaskManager.Task::(_updateState) Task=`4716234f-9e9d-4bfb-bba2-f9dd7361c7c4`::moving from state init -> state preparing >Thread-49::INFO::2014-10-05 21:22:23,919::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49::INFO::2014-10-05 21:22:23,920::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-49::DEBUG::2014-10-05 21:22:23,920::task::1191::Storage.TaskManager.Task::(prepare) Task=`4716234f-9e9d-4bfb-bba2-f9dd7361c7c4`::finished: {} >Thread-49::DEBUG::2014-10-05 21:22:23,920::task::595::Storage.TaskManager.Task::(_updateState) Task=`4716234f-9e9d-4bfb-bba2-f9dd7361c7c4`::moving from state preparing -> state finished >Thread-49::DEBUG::2014-10-05 21:22:23,920::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49::DEBUG::2014-10-05 21:22:23,920::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49::DEBUG::2014-10-05 21:22:23,920::task::993::Storage.TaskManager.Task::(_decref) Task=`4716234f-9e9d-4bfb-bba2-f9dd7361c7c4`::ref 0 aborting False >Thread-49::DEBUG::2014-10-05 21:22:23,923::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:22:24,028::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:22:24,029::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-50::DEBUG::2014-10-05 21:22:24,030::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:22:27,172::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:22:27,173::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-51::DEBUG::2014-10-05 21:22:27,174::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:22:30,264::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:22:30,265::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-52::DEBUG::2014-10-05 21:22:30,267::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:22:33,386::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:22:33,386::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-53::DEBUG::2014-10-05 21:22:33,387::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:22:36,475::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:22:36,476::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-54::DEBUG::2014-10-05 21:22:36,477::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:22:39,567::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:22:39,567::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-55::DEBUG::2014-10-05 21:22:39,569::task::595::Storage.TaskManager.Task::(_updateState) Task=`e2cf557d-e525-457c-9689-4c4d1a0595e7`::moving from state init -> state preparing >Thread-55::INFO::2014-10-05 21:22:39,569::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-55::INFO::2014-10-05 21:22:39,569::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-55::DEBUG::2014-10-05 21:22:39,569::task::1191::Storage.TaskManager.Task::(prepare) Task=`e2cf557d-e525-457c-9689-4c4d1a0595e7`::finished: {} >Thread-55::DEBUG::2014-10-05 21:22:39,569::task::595::Storage.TaskManager.Task::(_updateState) Task=`e2cf557d-e525-457c-9689-4c4d1a0595e7`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-10-05 21:22:39,569::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-10-05 21:22:39,570::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-10-05 21:22:39,570::task::993::Storage.TaskManager.Task::(_decref) Task=`e2cf557d-e525-457c-9689-4c4d1a0595e7`::ref 0 aborting False >Thread-55::DEBUG::2014-10-05 21:22:39,571::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:22:39,664::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:22:39,665::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-56::DEBUG::2014-10-05 21:22:39,666::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:22:42,866::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:22:42,867::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-57::DEBUG::2014-10-05 21:22:42,868::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:22:45,956::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:22:45,957::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-58::DEBUG::2014-10-05 21:22:45,959::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:22:49,051::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:22:49,052::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-59::DEBUG::2014-10-05 21:22:49,053::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:22:52,142::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:22:52,143::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-60::DEBUG::2014-10-05 21:22:52,144::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:22:55,234::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:22:55,235::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-61::DEBUG::2014-10-05 21:22:55,236::task::595::Storage.TaskManager.Task::(_updateState) Task=`df1ef23d-7578-4e6f-bdaa-1357f76fefb8`::moving from state init -> state preparing >Thread-61::INFO::2014-10-05 21:22:55,236::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-61::INFO::2014-10-05 21:22:55,237::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-61::DEBUG::2014-10-05 21:22:55,237::task::1191::Storage.TaskManager.Task::(prepare) Task=`df1ef23d-7578-4e6f-bdaa-1357f76fefb8`::finished: {} >Thread-61::DEBUG::2014-10-05 21:22:55,237::task::595::Storage.TaskManager.Task::(_updateState) Task=`df1ef23d-7578-4e6f-bdaa-1357f76fefb8`::moving from state preparing -> state finished >Thread-61::DEBUG::2014-10-05 21:22:55,237::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-61::DEBUG::2014-10-05 21:22:55,237::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-61::DEBUG::2014-10-05 21:22:55,237::task::993::Storage.TaskManager.Task::(_decref) Task=`df1ef23d-7578-4e6f-bdaa-1357f76fefb8`::ref 0 aborting False >Thread-61::DEBUG::2014-10-05 21:22:55,238::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:22:55,330::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:22:55,331::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-62::DEBUG::2014-10-05 21:22:55,332::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:22:58,514::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:22:58,515::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-63::DEBUG::2014-10-05 21:22:58,516::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:23:01,603::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:23:01,604::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-64::DEBUG::2014-10-05 21:23:01,605::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:23:04,694::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:23:04,695::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-65::DEBUG::2014-10-05 21:23:04,696::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:23:07,782::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:23:07,783::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-66::DEBUG::2014-10-05 21:23:07,785::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:23:10,871::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:23:10,872::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-67::DEBUG::2014-10-05 21:23:10,874::task::595::Storage.TaskManager.Task::(_updateState) Task=`fbaffdf6-6fbc-4776-a31b-d03f6efe6df1`::moving from state init -> state preparing >Thread-67::INFO::2014-10-05 21:23:10,874::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-67::INFO::2014-10-05 21:23:10,874::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-67::DEBUG::2014-10-05 21:23:10,874::task::1191::Storage.TaskManager.Task::(prepare) Task=`fbaffdf6-6fbc-4776-a31b-d03f6efe6df1`::finished: {} >Thread-67::DEBUG::2014-10-05 21:23:10,874::task::595::Storage.TaskManager.Task::(_updateState) Task=`fbaffdf6-6fbc-4776-a31b-d03f6efe6df1`::moving from state preparing -> state finished >Thread-67::DEBUG::2014-10-05 21:23:10,874::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-67::DEBUG::2014-10-05 21:23:10,874::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-67::DEBUG::2014-10-05 21:23:10,874::task::993::Storage.TaskManager.Task::(_decref) Task=`fbaffdf6-6fbc-4776-a31b-d03f6efe6df1`::ref 0 aborting False >Thread-67::DEBUG::2014-10-05 21:23:10,876::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:23:10,969::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:23:10,970::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-68::DEBUG::2014-10-05 21:23:10,971::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:23:14,203::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:23:14,204::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-69::DEBUG::2014-10-05 21:23:14,205::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:23:17,293::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:23:17,294::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-70::DEBUG::2014-10-05 21:23:17,295::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:23:20,389::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:23:20,390::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-71::DEBUG::2014-10-05 21:23:20,392::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:23:23,482::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:23:23,482::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-72::DEBUG::2014-10-05 21:23:23,483::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:23:26,570::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:23:26,571::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-73::DEBUG::2014-10-05 21:23:26,573::task::595::Storage.TaskManager.Task::(_updateState) Task=`f28a97ec-ae48-48a1-a29c-8a431e420aba`::moving from state init -> state preparing >Thread-73::INFO::2014-10-05 21:23:26,573::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-73::INFO::2014-10-05 21:23:26,573::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-73::DEBUG::2014-10-05 21:23:26,574::task::1191::Storage.TaskManager.Task::(prepare) Task=`f28a97ec-ae48-48a1-a29c-8a431e420aba`::finished: {} >Thread-73::DEBUG::2014-10-05 21:23:26,574::task::595::Storage.TaskManager.Task::(_updateState) Task=`f28a97ec-ae48-48a1-a29c-8a431e420aba`::moving from state preparing -> state finished >Thread-73::DEBUG::2014-10-05 21:23:26,574::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-73::DEBUG::2014-10-05 21:23:26,574::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-73::DEBUG::2014-10-05 21:23:26,574::task::993::Storage.TaskManager.Task::(_decref) Task=`f28a97ec-ae48-48a1-a29c-8a431e420aba`::ref 0 aborting False >Thread-73::DEBUG::2014-10-05 21:23:26,576::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:23:26,668::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:23:26,669::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-74::DEBUG::2014-10-05 21:23:26,670::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:23:29,822::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:23:29,823::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-75::DEBUG::2014-10-05 21:23:29,824::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:23:32,910::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:23:32,911::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-76::DEBUG::2014-10-05 21:23:32,912::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:23:36,006::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:23:36,007::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-77::DEBUG::2014-10-05 21:23:36,008::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:23:39,096::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:23:39,097::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-78::DEBUG::2014-10-05 21:23:39,099::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:23:42,186::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:23:42,187::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-79::DEBUG::2014-10-05 21:23:42,188::task::595::Storage.TaskManager.Task::(_updateState) Task=`97ef2563-244c-46af-915e-806e6573201b`::moving from state init -> state preparing >Thread-79::INFO::2014-10-05 21:23:42,188::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-79::INFO::2014-10-05 21:23:42,188::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-79::DEBUG::2014-10-05 21:23:42,189::task::1191::Storage.TaskManager.Task::(prepare) Task=`97ef2563-244c-46af-915e-806e6573201b`::finished: {} >Thread-79::DEBUG::2014-10-05 21:23:42,189::task::595::Storage.TaskManager.Task::(_updateState) Task=`97ef2563-244c-46af-915e-806e6573201b`::moving from state preparing -> state finished >Thread-79::DEBUG::2014-10-05 21:23:42,189::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-79::DEBUG::2014-10-05 21:23:42,189::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-79::DEBUG::2014-10-05 21:23:42,189::task::993::Storage.TaskManager.Task::(_decref) Task=`97ef2563-244c-46af-915e-806e6573201b`::ref 0 aborting False >Thread-79::DEBUG::2014-10-05 21:23:42,190::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:23:42,282::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:23:42,283::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-80::DEBUG::2014-10-05 21:23:42,284::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:23:45,553::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:23:45,554::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-81::DEBUG::2014-10-05 21:23:45,555::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:23:48,644::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:23:48,649::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-82::DEBUG::2014-10-05 21:23:48,650::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >MainThread::DEBUG::2014-10-05 21:23:48,817::vdsm::58::vds::(sigtermHandler) Received signal 15 >MainThread::DEBUG::2014-10-05 21:23:48,818::protocoldetector::135::vds.MultiProtocolAcceptor::(stop) Stopping Acceptor >MainThread::INFO::2014-10-05 21:23:48,818::__init__::563::jsonrpc.JsonRpcServer::(stop) Stopping JsonRPC Server >Detector thread::DEBUG::2014-10-05 21:23:48,818::protocoldetector::106::vds.MultiProtocolAcceptor::(_cleanup) Cleaning Acceptor >MainThread::INFO::2014-10-05 21:23:48,819::vmchannels::188::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2014-10-05 21:23:48,819::momIF::91::MOM::(stop) Shutting down MOM >MainThread::DEBUG::2014-10-05 21:23:48,820::task::595::Storage.TaskManager.Task::(_updateState) Task=`5965e340-b26d-4bb3-8d31-916f7417d12b`::moving from state init -> state preparing >MainThread::INFO::2014-10-05 21:23:48,820::logUtils::44::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2014-10-05 21:23:48,821::storageServer::706::Storage.ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2014-10-05 21:23:48,823::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/04511d19-f03a-469a-9e31-ab51d6e75a56/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:23:48,824::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/67ba7199-44ec-4562-9c17-ebc3903c7c7f/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:23:48,824::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/83d26f74-2a29-4025-8210-5ab61cb49a82/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:23:48,825::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/fd98c9a9-23f9-43f5-8e7b-4022f5bdd683/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:23:48,825::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/b9c2b835-cdbd-48ea-8d69-7a33042c79b1/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:23:48,825::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/c175b488-c93a-4dff-b709-b137853eab67/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:23:48,826::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/6ca7f5aa-6229-4549-9c6f-54ec7b7099a6/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:23:48,826::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/681bd5bc-4d98-4da3-bd9a-cd37eec7b9af/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:23:48,827::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050/master` is not mounted, skipping >MainThread::INFO::2014-10-05 21:23:48,869::domainMonitor::137::Storage.DomainMonitor::(close) Stopping domain monitors >MainThread::DEBUG::2014-10-05 21:23:48,869::taskManager::90::Storage.TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2014-10-05 21:23:48,869::logUtils::47::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2014-10-05 21:23:48,869::task::1191::Storage.TaskManager.Task::(prepare) Task=`5965e340-b26d-4bb3-8d31-916f7417d12b`::finished: None >MainThread::DEBUG::2014-10-05 21:23:48,869::task::595::Storage.TaskManager.Task::(_updateState) Task=`5965e340-b26d-4bb3-8d31-916f7417d12b`::moving from state preparing -> state finished >MainThread::DEBUG::2014-10-05 21:23:48,869::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2014-10-05 21:23:48,869::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2014-10-05 21:23:48,869::task::993::Storage.TaskManager.Task::(_decref) Task=`5965e340-b26d-4bb3-8d31-916f7417d12b`::ref 0 aborting False >MainThread::INFO::2014-10-05 21:23:48,869::vdsm::137::vds::(run) VDSM main thread ended. Waiting for 15 other threads... >MainThread::INFO::2014-10-05 21:23:48,869::vdsm::141::vds::(run) <_MainThread(MainThread, started 140531455411968)> >MainThread::INFO::2014-10-05 21:23:48,869::vdsm::141::vds::(run) <Thread(libvirtEventLoop, started daemon 140531367085824)> >MainThread::INFO::2014-10-05 21:23:48,869::vdsm::141::vds::(run) <PolicyEngine(PolicyEngine, started daemon 140529585092352)> >MainThread::INFO::2014-10-05 21:23:48,869::vdsm::141::vds::(run) <WorkerThread(Thread-8, started daemon 140531076146944)> >MainThread::INFO::2014-10-05 21:23:48,869::vdsm::141::vds::(run) <WorkerThread(Thread-4, started daemon 140531118106368)> >MainThread::INFO::2014-10-05 21:23:48,870::vdsm::141::vds::(run) <WorkerThread(Thread-6, started daemon 140531097126656)> >MainThread::INFO::2014-10-05 21:23:48,870::vdsm::141::vds::(run) <WorkerThread(Thread-5, started daemon 140531107616512)> >MainThread::INFO::2014-10-05 21:23:48,870::vdsm::141::vds::(run) <WorkerThread(Thread-10, started daemon 140530715453184)> >MainThread::INFO::2014-10-05 21:23:48,870::vdsm::141::vds::(run) <WorkerThread(Thread-3, started daemon 140531128596224)> >MainThread::INFO::2014-10-05 21:23:48,870::vmchannels::188::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2014-10-05 21:23:48,870::vdsm::141::vds::(run) <Listener(VM Channels Listener, started daemon 140530345051904)> >MainThread::INFO::2014-10-05 21:23:48,870::vdsm::141::vds::(run) <WorkerThread(Thread-7, started daemon 140531086636800)> >MainThread::INFO::2014-10-05 21:23:48,870::momIF::91::MOM::(stop) Shutting down MOM >MainThread::INFO::2014-10-05 21:23:48,870::vdsm::141::vds::(run) <MomThread(MOM, started 140530366031616)> >MainThread::INFO::2014-10-05 21:23:48,870::vdsm::141::vds::(run) <WorkerThread(Thread-9, started daemon 140530725943040)> >MainThread::INFO::2014-10-05 21:23:48,870::vdsm::141::vds::(run) <WorkerThread(Thread-2, started daemon 140531346106112)> >MainThread::INFO::2014-10-05 21:23:48,870::vdsm::141::vds::(run) <WorkerThread(Thread-1, started daemon 140531356595968)> >MainThread::INFO::2014-10-05 21:23:48,870::vdsm::141::vds::(run) <GuestManager(GuestManager, started daemon 140530019145472)> >VM Channels Listener::INFO::2014-10-05 21:23:49,194::vmchannels::183::vds::(run) VM channels listener thread has ended. >MainThread::INFO::2014-10-05 21:24:32,722::vdsm::131::vds::(run) (PID: 3061) I am the actual vdsm 4.16.6-0.el6 voodoo1.tlv.redhat.com (2.6.32-431.29.2.el6.x86_64) >MainThread::DEBUG::2014-10-05 21:24:32,732::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2014-10-05 21:24:32,732::threadPool::35::Storage.ThreadPool::(__init__) Enter - numThreads: 10, waitTimeout: 3, maxTasks: 500 >MainThread::DEBUG::2014-10-05 21:24:32,733::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt >MainThread::WARNING::2014-10-05 21:24:32,734::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt already exists >MainThread::DEBUG::2014-10-05 21:24:32,741::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/04511d19-f03a-469a-9e31-ab51d6e75a56/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:24:32,742::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/67ba7199-44ec-4562-9c17-ebc3903c7c7f/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:24:32,742::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/83d26f74-2a29-4025-8210-5ab61cb49a82/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:24:32,742::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/fd98c9a9-23f9-43f5-8e7b-4022f5bdd683/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:24:32,743::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/b9c2b835-cdbd-48ea-8d69-7a33042c79b1/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:24:32,743::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/c175b488-c93a-4dff-b709-b137853eab67/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:24:32,744::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/6ca7f5aa-6229-4549-9c6f-54ec7b7099a6/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:24:32,744::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/681bd5bc-4d98-4da3-bd9a-cd37eec7b9af/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:24:32,744::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:24:32,799::supervdsm::77::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2014-10-05 21:24:32,810::multipath::142::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2014-10-05 21:24:32,811::hsm::418::Storage.Misc.excCmd::(__validateLvmLockingType) /usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type (cwd None) >MainThread::DEBUG::2014-10-05 21:24:32,906::hsm::418::Storage.Misc.excCmd::(__validateLvmLockingType) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2014-10-05 21:24:32,906::hsm::442::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >MainThread::DEBUG::2014-10-05 21:24:32,907::hsm::474::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >MainThread::DEBUG::2014-10-05 21:24:32,907::hsm::475::Storage.HSM::(__cleanStorageRepository) Mount list: [] >MainThread::DEBUG::2014-10-05 21:24:32,907::hsm::477::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >MainThread::DEBUG::2014-10-05 21:24:32,909::hsm::520::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >storageRefresh::DEBUG::2014-10-05 21:24:32,910::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2014-10-05 21:24:32,911::dispatcher::46::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >storageRefresh::DEBUG::2014-10-05 21:24:32,911::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >storageRefresh::DEBUG::2014-10-05 21:24:32,928::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >storageRefresh::DEBUG::2014-10-05 21:24:32,931::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >storageRefresh::DEBUG::2014-10-05 21:24:32,932::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >storageRefresh::DEBUG::2014-10-05 21:24:32,932::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >storageRefresh::DEBUG::2014-10-05 21:24:32,970::misc::751::Storage.SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2014-10-05 21:24:32,975::task::595::Storage.TaskManager.Task::(_updateState) Task=`d3ad5efb-2b71-41d9-8047-c25977f43be6`::moving from state init -> state preparing >MainThread::INFO::2014-10-05 21:24:32,976::logUtils::44::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback(callbackFunc=<bound method clientIF.contEIOVms of <clientIF.clientIF object at 0x2da2490>>) >MainThread::INFO::2014-10-05 21:24:32,976::logUtils::47::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback, Return response: None >MainThread::DEBUG::2014-10-05 21:24:32,976::task::1191::Storage.TaskManager.Task::(prepare) Task=`d3ad5efb-2b71-41d9-8047-c25977f43be6`::finished: None >MainThread::DEBUG::2014-10-05 21:24:32,976::task::595::Storage.TaskManager.Task::(_updateState) Task=`d3ad5efb-2b71-41d9-8047-c25977f43be6`::moving from state preparing -> state finished >MainThread::DEBUG::2014-10-05 21:24:32,976::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2014-10-05 21:24:32,976::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2014-10-05 21:24:32,976::task::993::Storage.TaskManager.Task::(_decref) Task=`d3ad5efb-2b71-41d9-8047-c25977f43be6`::ref 0 aborting False >storageRefresh::DEBUG::2014-10-05 21:24:32,977::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >MainThread::INFO::2014-10-05 21:24:32,982::momIF::49::MOM::(__init__) Starting up MOM >MainThread::INFO::2014-10-05 21:24:32,987::vmchannels::192::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2014-10-05 21:24:32,990::vmchannels::175::vds::(run) Starting VM channels listener thread. >clientIFinit::DEBUG::2014-10-05 21:24:32,991::libvirtconnection::150::root::(get) trying to connect libvirt >storageRefresh::DEBUG::2014-10-05 21:24:32,992::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >storageRefresh::DEBUG::2014-10-05 21:24:32,999::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >storageRefresh::DEBUG::2014-10-05 21:24:32,999::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-10-05 21:24:32,999::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >MainThread::DEBUG::2014-10-05 21:24:32,999::protocoldetector::131::vds.MultiProtocolAcceptor::(add_detector) adding detector: <rpc.BindingXMLRPC.XmlDetector instance at 0x2ea6488> >storageRefresh::DEBUG::2014-10-05 21:24:32,999::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-10-05 21:24:33,003::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >storageRefresh::DEBUG::2014-10-05 21:24:33,005::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-10-05 21:24:33,005::misc::751::Storage.SamplingMethod::(__call__) Returning last result >storageRefresh::DEBUG::2014-10-05 21:24:33,007::lvm::317::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >storageRefresh::DEBUG::2014-10-05 21:24:33,028::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm pvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size (cwd None) >MainThread::DEBUG::2014-10-05 21:24:33,106::protocoldetector::131::vds.MultiProtocolAcceptor::(add_detector) adding detector: <yajsonrpc.stompReactor.StompDetector instance at 0x3066b90> >JsonRpcServer::DEBUG::2014-10-05 21:24:33,107::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Detector thread::DEBUG::2014-10-05 21:24:33,108::protocoldetector::70::vds.MultiProtocolAcceptor::(serve_forever) Acceptor running >Detector thread::DEBUG::2014-10-05 21:24:33,108::protocoldetector::72::vds.MultiProtocolAcceptor::(serve_forever) Using required_size=7 >storageRefresh::DEBUG::2014-10-05 21:24:33,116::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >storageRefresh::DEBUG::2014-10-05 21:24:33,117::lvm::342::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >storageRefresh::DEBUG::2014-10-05 21:24:33,118::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >storageRefresh::DEBUG::2014-10-05 21:24:33,118::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >storageRefresh::DEBUG::2014-10-05 21:24:33,195::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-10-05 21:24:33,196::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >storageRefresh::DEBUG::2014-10-05 21:24:33,197::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags (cwd None) >storageRefresh::DEBUG::2014-10-05 21:24:33,267::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-10-05 21:24:33,267::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >storageRefresh::DEBUG::2014-10-05 21:24:33,268::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >storageRefresh::DEBUG::2014-10-05 21:24:33,339::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-10-05 21:24:33,339::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >storageRefresh::DEBUG::2014-10-05 21:24:33,339::hsm::387::Storage.HSM::(storageRefresh) HSM is ready >Detector thread::DEBUG::2014-10-05 21:25:51,330::protocoldetector::166::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.201.227:58550 >Detector thread::DEBUG::2014-10-05 21:25:51,410::protocoldetector::177::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.201.227:58550 >Detector thread::DEBUG::2014-10-05 21:25:51,411::protocoldetector::203::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol stomp from 10.35.201.227:58550 >Detector thread::DEBUG::2014-10-05 21:25:51,411::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='CONNECT'> >Detector thread::INFO::2014-10-05 21:25:51,411::stompReactor::66::Broker.StompAdapter::(_cmd_connect) Processing CONNECT request >Detector thread::DEBUG::2014-10-05 21:25:51,411::stompReactor::338::protocoldetector.StompDetector::(handleSocket) Stomp detected from ('10.35.201.227', 58550) >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:25:51,491::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SUBSCRIBE'> >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:25:51,492::stompReactor::89::Broker.StompAdapter::(_cmd_subscribe) Subscribe command ignored >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:25:51,609::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:25:51,610::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-13::DEBUG::2014-10-05 21:25:51,610::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'b86b687a-d073-497a-ac8a-249025419a3e'} >Thread-13::DEBUG::2014-10-05 21:25:51,611::task::595::Storage.TaskManager.Task::(_updateState) Task=`78a71559-d169-476a-acd4-ae8d6a4a595c`::moving from state init -> state preparing >Thread-13::INFO::2014-10-05 21:25:51,611::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'b86b687a-d073-497a-ac8a-249025419a3e', options=None) >Thread-13::ERROR::2014-10-05 21:25:51,611::task::866::Storage.TaskManager.Task::(_setError) Task=`78a71559-d169-476a-acd4-ae8d6a4a595c`::Unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/storage/task.py", line 873, in _run > return fn(*args, **kargs) > File "/usr/share/vdsm/logUtils.py", line 45, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/storage/hsm.py", line 609, in getSpmStatus > pool = self.getPool(spUUID) > File "/usr/share/vdsm/storage/hsm.py", line 325, in getPool > raise se.StoragePoolUnknown(spUUID) >StoragePoolUnknown: Unknown pool id, pool not connected: (u'b86b687a-d073-497a-ac8a-249025419a3e',) >Thread-13::DEBUG::2014-10-05 21:25:51,613::task::885::Storage.TaskManager.Task::(_run) Task=`78a71559-d169-476a-acd4-ae8d6a4a595c`::Task._run: 78a71559-d169-476a-acd4-ae8d6a4a595c (u'b86b687a-d073-497a-ac8a-249025419a3e',) {} failed - stopping task >Thread-13::DEBUG::2014-10-05 21:25:51,613::task::1217::Storage.TaskManager.Task::(stop) Task=`78a71559-d169-476a-acd4-ae8d6a4a595c`::stopping in state preparing (force False) >Thread-13::DEBUG::2014-10-05 21:25:51,613::task::993::Storage.TaskManager.Task::(_decref) Task=`78a71559-d169-476a-acd4-ae8d6a4a595c`::ref 1 aborting True >Thread-13::INFO::2014-10-05 21:25:51,613::task::1171::Storage.TaskManager.Task::(prepare) Task=`78a71559-d169-476a-acd4-ae8d6a4a595c`::aborting: Task is aborted: 'Unknown pool id, pool not connected' - code 309 >Thread-13::DEBUG::2014-10-05 21:25:51,613::task::1176::Storage.TaskManager.Task::(prepare) Task=`78a71559-d169-476a-acd4-ae8d6a4a595c`::Prepare: aborted: Unknown pool id, pool not connected >Thread-13::DEBUG::2014-10-05 21:25:51,613::task::993::Storage.TaskManager.Task::(_decref) Task=`78a71559-d169-476a-acd4-ae8d6a4a595c`::ref 0 aborting True >Thread-13::DEBUG::2014-10-05 21:25:51,613::task::928::Storage.TaskManager.Task::(_doAbort) Task=`78a71559-d169-476a-acd4-ae8d6a4a595c`::Task._doAbort: force False >Thread-13::DEBUG::2014-10-05 21:25:51,613::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-10-05 21:25:51,613::task::595::Storage.TaskManager.Task::(_updateState) Task=`78a71559-d169-476a-acd4-ae8d6a4a595c`::moving from state preparing -> state aborting >Thread-13::DEBUG::2014-10-05 21:25:51,613::task::550::Storage.TaskManager.Task::(__state_aborting) Task=`78a71559-d169-476a-acd4-ae8d6a4a595c`::_aborting: recover policy none >Thread-13::DEBUG::2014-10-05 21:25:51,613::task::595::Storage.TaskManager.Task::(_updateState) Task=`78a71559-d169-476a-acd4-ae8d6a4a595c`::moving from state aborting -> state failed >Thread-13::DEBUG::2014-10-05 21:25:51,613::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-10-05 21:25:51,613::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::ERROR::2014-10-05 21:25:51,614::dispatcher::76::Storage.Dispatcher::(wrapper) {'status': {'message': "Unknown pool id, pool not connected: (u'b86b687a-d073-497a-ac8a-249025419a3e',)", 'code': 309}} >Thread-13::DEBUG::2014-10-05 21:25:51,614::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:25:51,709::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:25:51,710::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-14::DEBUG::2014-10-05 21:25:51,710::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.connect' in bridge with {u'masterVersion': 5, u'domainDict': {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': u'active', u'ff559f46-c495-4f6b-901c-2a624042a050': u'active'}, u'storagepoolID': u'b86b687a-d073-497a-ac8a-249025419a3e', u'scsiKey': u'b86b687a-d073-497a-ac8a-249025419a3e', u'masterSdUUID': u'ff559f46-c495-4f6b-901c-2a624042a050', u'hostID': 3} >Thread-14::DEBUG::2014-10-05 21:25:51,710::task::595::Storage.TaskManager.Task::(_updateState) Task=`a620633f-05a0-45e3-92f7-c62816a86133`::moving from state init -> state preparing >Thread-14::INFO::2014-10-05 21:25:51,710::logUtils::44::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID=u'b86b687a-d073-497a-ac8a-249025419a3e', hostID=3, msdUUID=u'ff559f46-c495-4f6b-901c-2a624042a050', masterVersion=5, domainsMap={u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': u'active', u'ff559f46-c495-4f6b-901c-2a624042a050': u'active'}, options=None) >Thread-14::DEBUG::2014-10-05 21:25:51,711::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.HsmDomainMonitorLock`ReqID=`bead925d-8204-4d8e-99e2-8bb1613dd779`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1011' at 'connectStoragePool' >Thread-14::DEBUG::2014-10-05 21:25:51,711::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.HsmDomainMonitorLock' for lock type 'exclusive' >Thread-14::DEBUG::2014-10-05 21:25:51,711::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.HsmDomainMonitorLock' is free. Now locking as 'exclusive' (1 active user) >Thread-14::DEBUG::2014-10-05 21:25:51,711::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.HsmDomainMonitorLock`ReqID=`bead925d-8204-4d8e-99e2-8bb1613dd779`::Granted request >Thread-14::DEBUG::2014-10-05 21:25:51,712::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.b86b687a-d073-497a-ac8a-249025419a3e`ReqID=`823836ea-92ed-4003-8308-db0a6eb41475`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1059' at '_connectStoragePool' >Thread-14::DEBUG::2014-10-05 21:25:51,712::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.b86b687a-d073-497a-ac8a-249025419a3e' for lock type 'exclusive' >Thread-14::DEBUG::2014-10-05 21:25:51,712::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.b86b687a-d073-497a-ac8a-249025419a3e' is free. Now locking as 'exclusive' (1 active user) >Thread-14::DEBUG::2014-10-05 21:25:51,712::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.b86b687a-d073-497a-ac8a-249025419a3e`ReqID=`823836ea-92ed-4003-8308-db0a6eb41475`::Granted request >Thread-14::DEBUG::2014-10-05 21:25:51,712::spbackends::485::Storage.StoragePoolMemoryBackend::(updateVersionAndDomains) updating domain version to 5 and domains map to {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': u'active', u'ff559f46-c495-4f6b-901c-2a624042a050': u'active'} >Thread-14::INFO::2014-10-05 21:25:51,712::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 5 and domains map {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': u'Active', u'ff559f46-c495-4f6b-901c-2a624042a050': u'Active'} >Thread-14::INFO::2014-10-05 21:25:51,712::sp::133::Storage.StoragePool::(setBackend) updating pool b86b687a-d073-497a-ac8a-249025419a3e backend from type NoneType instance 0x3a11372210 to type StoragePoolMemoryBackend instance 0x7f6fd41b98e8 >Thread-14::INFO::2014-10-05 21:25:51,712::sp::631::Storage.StoragePool::(connect) Connect host #3 to the storage pool b86b687a-d073-497a-ac8a-249025419a3e with master domain: ff559f46-c495-4f6b-901c-2a624042a050 (ver = 5) >Thread-14::DEBUG::2014-10-05 21:25:51,712::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-14::DEBUG::2014-10-05 21:25:51,713::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-14::DEBUG::2014-10-05 21:25:51,713::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-14::DEBUG::2014-10-05 21:25:51,713::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-14::DEBUG::2014-10-05 21:25:51,713::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-14::DEBUG::2014-10-05 21:25:51,713::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-14::DEBUG::2014-10-05 21:25:51,713::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-14::DEBUG::2014-10-05 21:25:51,713::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-14::DEBUG::2014-10-05 21:25:51,713::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-14::DEBUG::2014-10-05 21:25:51,713::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-14::DEBUG::2014-10-05 21:25:51,713::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-14::DEBUG::2014-10-05 21:25:51,713::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-14::DEBUG::2014-10-05 21:25:51,726::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-14::DEBUG::2014-10-05 21:25:51,733::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-14::DEBUG::2014-10-05 21:25:51,742::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2014-10-05 21:25:51,743::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-14::DEBUG::2014-10-05 21:25:51,743::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-14::DEBUG::2014-10-05 21:25:51,743::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-14::DEBUG::2014-10-05 21:25:51,743::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-14::DEBUG::2014-10-05 21:25:51,743::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-14::DEBUG::2014-10-05 21:25:51,743::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-14::DEBUG::2014-10-05 21:25:51,743::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-14::ERROR::2014-10-05 21:25:51,743::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain ff559f46-c495-4f6b-901c-2a624042a050 >Thread-14::ERROR::2014-10-05 21:25:51,743::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain ff559f46-c495-4f6b-901c-2a624042a050 >Thread-14::DEBUG::2014-10-05 21:25:51,743::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-14::DEBUG::2014-10-05 21:25:51,744::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name ff559f46-c495-4f6b-901c-2a624042a050 (cwd None) >Thread-14::DEBUG::2014-10-05 21:25:51,827::lvm::288::Storage.Misc.excCmd::(cmd) FAILED: <err> = ' Volume group "ff559f46-c495-4f6b-901c-2a624042a050" not found\n Skipping volume group ff559f46-c495-4f6b-901c-2a624042a050\n'; <rc> = 5 >Thread-14::WARNING::2014-10-05 21:25:51,828::lvm::370::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "ff559f46-c495-4f6b-901c-2a624042a050" not found', ' Skipping volume group ff559f46-c495-4f6b-901c-2a624042a050'] >Thread-14::DEBUG::2014-10-05 21:25:51,828::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-14::ERROR::2014-10-05 21:25:51,829::sdc::143::Storage.StorageDomainCache::(_findDomain) domain ff559f46-c495-4f6b-901c-2a624042a050 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 171, in _findUnfetchedDomain > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: (u'ff559f46-c495-4f6b-901c-2a624042a050',) >Thread-14::DEBUG::2014-10-05 21:25:51,830::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.b86b687a-d073-497a-ac8a-249025419a3e' >Thread-14::DEBUG::2014-10-05 21:25:51,830::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.b86b687a-d073-497a-ac8a-249025419a3e' (0 active users) >Thread-14::DEBUG::2014-10-05 21:25:51,830::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.b86b687a-d073-497a-ac8a-249025419a3e' is free, finding out if anyone is waiting for it. >Thread-14::DEBUG::2014-10-05 21:25:51,830::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.b86b687a-d073-497a-ac8a-249025419a3e', Clearing records. >Thread-14::DEBUG::2014-10-05 21:25:51,830::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.HsmDomainMonitorLock' >Thread-14::DEBUG::2014-10-05 21:25:51,830::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.HsmDomainMonitorLock' (0 active users) >Thread-14::DEBUG::2014-10-05 21:25:51,830::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.HsmDomainMonitorLock' is free, finding out if anyone is waiting for it. >Thread-14::DEBUG::2014-10-05 21:25:51,830::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.HsmDomainMonitorLock', Clearing records. >Thread-14::ERROR::2014-10-05 21:25:51,830::task::866::Storage.TaskManager.Task::(_setError) Task=`a620633f-05a0-45e3-92f7-c62816a86133`::Unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/storage/task.py", line 873, in _run > return fn(*args, **kargs) > File "/usr/share/vdsm/logUtils.py", line 45, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/storage/hsm.py", line 1013, in connectStoragePool > spUUID, hostID, msdUUID, masterVersion, domainsMap) > File "/usr/share/vdsm/storage/hsm.py", line 1084, in _connectStoragePool > res = pool.connect(hostID, msdUUID, masterVersion) > File "/usr/share/vdsm/storage/sp.py", line 638, in connect > self.__rebuild(msdUUID=msdUUID, masterVersion=masterVersion) > File "/usr/share/vdsm/storage/sp.py", line 1164, in __rebuild > self.setMasterDomain(msdUUID, masterVersion) > File "/usr/share/vdsm/storage/sp.py", line 1375, in setMasterDomain > raise se.StoragePoolMasterNotFound(self.spUUID, msdUUID) >StoragePoolMasterNotFound: Cannot find master domain: u'spUUID=b86b687a-d073-497a-ac8a-249025419a3e, msdUUID=ff559f46-c495-4f6b-901c-2a624042a050' >Thread-14::DEBUG::2014-10-05 21:25:51,831::task::885::Storage.TaskManager.Task::(_run) Task=`a620633f-05a0-45e3-92f7-c62816a86133`::Task._run: a620633f-05a0-45e3-92f7-c62816a86133 (u'b86b687a-d073-497a-ac8a-249025419a3e', 3, u'ff559f46-c495-4f6b-901c-2a624042a050', 5, {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': u'active', u'ff559f46-c495-4f6b-901c-2a624042a050': u'active'}) {} failed - stopping task >Thread-14::DEBUG::2014-10-05 21:25:51,831::task::1217::Storage.TaskManager.Task::(stop) Task=`a620633f-05a0-45e3-92f7-c62816a86133`::stopping in state preparing (force False) >Thread-14::DEBUG::2014-10-05 21:25:51,831::task::993::Storage.TaskManager.Task::(_decref) Task=`a620633f-05a0-45e3-92f7-c62816a86133`::ref 1 aborting True >Thread-14::INFO::2014-10-05 21:25:51,831::task::1171::Storage.TaskManager.Task::(prepare) Task=`a620633f-05a0-45e3-92f7-c62816a86133`::aborting: Task is aborted: 'Cannot find master domain' - code 304 >Thread-14::DEBUG::2014-10-05 21:25:51,832::task::1176::Storage.TaskManager.Task::(prepare) Task=`a620633f-05a0-45e3-92f7-c62816a86133`::Prepare: aborted: Cannot find master domain >Thread-14::DEBUG::2014-10-05 21:25:51,832::task::993::Storage.TaskManager.Task::(_decref) Task=`a620633f-05a0-45e3-92f7-c62816a86133`::ref 0 aborting True >Thread-14::DEBUG::2014-10-05 21:25:51,832::task::928::Storage.TaskManager.Task::(_doAbort) Task=`a620633f-05a0-45e3-92f7-c62816a86133`::Task._doAbort: force False >Thread-14::DEBUG::2014-10-05 21:25:51,832::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-10-05 21:25:51,832::task::595::Storage.TaskManager.Task::(_updateState) Task=`a620633f-05a0-45e3-92f7-c62816a86133`::moving from state preparing -> state aborting >Thread-14::DEBUG::2014-10-05 21:25:51,832::task::550::Storage.TaskManager.Task::(__state_aborting) Task=`a620633f-05a0-45e3-92f7-c62816a86133`::_aborting: recover policy none >Thread-14::DEBUG::2014-10-05 21:25:51,832::task::595::Storage.TaskManager.Task::(_updateState) Task=`a620633f-05a0-45e3-92f7-c62816a86133`::moving from state aborting -> state failed >Thread-14::DEBUG::2014-10-05 21:25:51,832::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-10-05 21:25:51,832::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::ERROR::2014-10-05 21:25:51,832::dispatcher::76::Storage.Dispatcher::(wrapper) {'status': {'message': "Cannot find master domain: u'spUUID=b86b687a-d073-497a-ac8a-249025419a3e, msdUUID=ff559f46-c495-4f6b-901c-2a624042a050'", 'code': 304}} >Thread-14::DEBUG::2014-10-05 21:25:51,832::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:25:51,923::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:25:51,924::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-15::DEBUG::2014-10-05 21:25:51,924::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'b86b687a-d073-497a-ac8a-249025419a3e'} >Thread-15::DEBUG::2014-10-05 21:25:51,925::task::595::Storage.TaskManager.Task::(_updateState) Task=`68b56a36-2d61-48d6-bb5a-133d1d7449de`::moving from state init -> state preparing >Thread-15::INFO::2014-10-05 21:25:51,925::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'b86b687a-d073-497a-ac8a-249025419a3e', options=None) >Thread-15::ERROR::2014-10-05 21:25:51,925::task::866::Storage.TaskManager.Task::(_setError) Task=`68b56a36-2d61-48d6-bb5a-133d1d7449de`::Unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/storage/task.py", line 873, in _run > return fn(*args, **kargs) > File "/usr/share/vdsm/logUtils.py", line 45, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/storage/hsm.py", line 609, in getSpmStatus > pool = self.getPool(spUUID) > File "/usr/share/vdsm/storage/hsm.py", line 325, in getPool > raise se.StoragePoolUnknown(spUUID) >StoragePoolUnknown: Unknown pool id, pool not connected: (u'b86b687a-d073-497a-ac8a-249025419a3e',) >Thread-15::DEBUG::2014-10-05 21:25:51,925::task::885::Storage.TaskManager.Task::(_run) Task=`68b56a36-2d61-48d6-bb5a-133d1d7449de`::Task._run: 68b56a36-2d61-48d6-bb5a-133d1d7449de (u'b86b687a-d073-497a-ac8a-249025419a3e',) {} failed - stopping task >Thread-15::DEBUG::2014-10-05 21:25:51,925::task::1217::Storage.TaskManager.Task::(stop) Task=`68b56a36-2d61-48d6-bb5a-133d1d7449de`::stopping in state preparing (force False) >Thread-15::DEBUG::2014-10-05 21:25:51,925::task::993::Storage.TaskManager.Task::(_decref) Task=`68b56a36-2d61-48d6-bb5a-133d1d7449de`::ref 1 aborting True >Thread-15::INFO::2014-10-05 21:25:51,925::task::1171::Storage.TaskManager.Task::(prepare) Task=`68b56a36-2d61-48d6-bb5a-133d1d7449de`::aborting: Task is aborted: 'Unknown pool id, pool not connected' - code 309 >Thread-15::DEBUG::2014-10-05 21:25:51,925::task::1176::Storage.TaskManager.Task::(prepare) Task=`68b56a36-2d61-48d6-bb5a-133d1d7449de`::Prepare: aborted: Unknown pool id, pool not connected >Thread-15::DEBUG::2014-10-05 21:25:51,925::task::993::Storage.TaskManager.Task::(_decref) Task=`68b56a36-2d61-48d6-bb5a-133d1d7449de`::ref 0 aborting True >Thread-15::DEBUG::2014-10-05 21:25:51,925::task::928::Storage.TaskManager.Task::(_doAbort) Task=`68b56a36-2d61-48d6-bb5a-133d1d7449de`::Task._doAbort: force False >Thread-15::DEBUG::2014-10-05 21:25:51,925::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-15::DEBUG::2014-10-05 21:25:51,926::task::595::Storage.TaskManager.Task::(_updateState) Task=`68b56a36-2d61-48d6-bb5a-133d1d7449de`::moving from state preparing -> state aborting >Thread-15::DEBUG::2014-10-05 21:25:51,926::task::550::Storage.TaskManager.Task::(__state_aborting) Task=`68b56a36-2d61-48d6-bb5a-133d1d7449de`::_aborting: recover policy none >Thread-15::DEBUG::2014-10-05 21:25:51,926::task::595::Storage.TaskManager.Task::(_updateState) Task=`68b56a36-2d61-48d6-bb5a-133d1d7449de`::moving from state aborting -> state failed >Thread-15::DEBUG::2014-10-05 21:25:51,926::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-15::DEBUG::2014-10-05 21:25:51,926::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-15::ERROR::2014-10-05 21:25:51,926::dispatcher::76::Storage.Dispatcher::(wrapper) {'status': {'message': "Unknown pool id, pool not connected: (u'b86b687a-d073-497a-ac8a-249025419a3e',)", 'code': 309}} >Thread-15::DEBUG::2014-10-05 21:25:51,926::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Detector thread::DEBUG::2014-10-05 21:30:01,864::protocoldetector::166::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.201.227:58577 >Detector thread::DEBUG::2014-10-05 21:30:01,945::protocoldetector::177::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.201.227:58577 >Detector thread::DEBUG::2014-10-05 21:30:01,946::protocoldetector::203::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol stomp from 10.35.201.227:58577 >Detector thread::DEBUG::2014-10-05 21:30:01,946::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='CONNECT'> >Detector thread::INFO::2014-10-05 21:30:01,946::stompReactor::66::Broker.StompAdapter::(_cmd_connect) Processing CONNECT request >Detector thread::DEBUG::2014-10-05 21:30:01,946::stompReactor::338::protocoldetector.StompDetector::(handleSocket) Stomp detected from ('10.35.201.227', 58577) >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:30:02,073::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SUBSCRIBE'> >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:30:02,073::stompReactor::89::Broker.StompAdapter::(_cmd_subscribe) Subscribe command ignored >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:30:02,477::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:30:02,479::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-16::DEBUG::2014-10-05 21:30:02,479::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getCapabilities' in bridge with {} >Thread-16::DEBUG::2014-10-05 21:30:02,577::utils::738::root::(execCmd) /sbin/ip route show to 0.0.0.0/0 table all (cwd None) >Thread-16::DEBUG::2014-10-05 21:30:02,584::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-10-05 21:30:02,655::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/bonding_masters (cwd None) >Thread-16::DEBUG::2014-10-05 21:30:02,674::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-10-05 21:30:02,676::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/wQEBFC9cmobjPqi/bonding/mode (cwd None) >Thread-16::DEBUG::2014-10-05 21:30:02,684::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-10-05 21:30:02,689::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/wQEBFC9cmobjPqi/bonding/mode (cwd None) >Thread-16::DEBUG::2014-10-05 21:30:02,701::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-10-05 21:30:02,707::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/wQEBFC9cmobjPqi/bonding/mode (cwd None) >Thread-16::DEBUG::2014-10-05 21:30:02,715::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-10-05 21:30:02,721::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/wQEBFC9cmobjPqi/bonding/mode (cwd None) >Thread-16::DEBUG::2014-10-05 21:30:02,731::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-10-05 21:30:02,735::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/wQEBFC9cmobjPqi/bonding/mode (cwd None) >Thread-16::DEBUG::2014-10-05 21:30:02,749::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-10-05 21:30:02,752::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/wQEBFC9cmobjPqi/bonding/mode (cwd None) >Thread-16::DEBUG::2014-10-05 21:30:02,762::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-10-05 21:30:02,766::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/wQEBFC9cmobjPqi/bonding/mode (cwd None) >Thread-16::DEBUG::2014-10-05 21:30:02,774::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-10-05 21:30:02,777::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/bonding_masters (cwd None) >Thread-16::DEBUG::2014-10-05 21:30:02,816::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-10-05 21:30:02,878::caps::646::root::(get) VirtioRNG DISABLED: libvirt version 0.10.2-29.el6_5.12 required >= 0.10.2-31 >Thread-16::DEBUG::2014-10-05 21:30:02,879::utils::738::root::(execCmd) numactl --hardware (cwd None) >Thread-16::DEBUG::2014-10-05 21:30:02,884::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-10-05 21:30:02,886::utils::738::root::(execCmd) sysctl -n -e kernel.numa_balancing (cwd None) >Thread-16::DEBUG::2014-10-05 21:30:02,890::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-10-05 21:30:02,907::caps::269::root::(getLiveMergeSupport) libvirt is missing 'VIR_DOMAIN_BLOCK_COMMIT_RELATIVE': live merge disabled >Thread-16::DEBUG::2014-10-05 21:30:02,910::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.getCapabilities' in bridge with {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:nsoffer1'}], 'FC': []}, 'packages2': {'kernel': {'release': '431.29.2.el6.x86_64', 'buildtime': 1406465746.0, 'version': '2.6.32'}, 'spice-server': {'release': '6.el6_5.2', 'buildtime': 1401730350L, 'version': '0.12.4'}, 'vdsm': {'release': '0.el6', 'buildtime': 1412232469L, 'version': '4.16.6'}, 'qemu-kvm': {'release': '2.415.el6_5.14', 'buildtime': 1408575896L, 'version': '0.12.1.2'}, 'libvirt': {'release': '29.el6_5.12', 'buildtime': 1408610144L, 'version': '0.10.2'}, 'qemu-img': {'release': '2.415.el6_5.14', 'buildtime': 1408575896L, 'version': '0.12.1.2'}, 'mom': {'release': '0.0.master.20140911.gitcf7ff17.el6', 'buildtime': 1410462635L, 'version': '0.4.2'}}, 'numaNodeDistance': {'0': [10]}, 'cpuModel': 'Intel Core Processor (Haswell)', 'liveMerge': 'false', 'hooks': {}, 'cpuSockets': '8', 'vmTypes': ['kvm'], 'selinux': {'mode': '1'}, 'kdumpStatus': 1, 'supportedProtocols': ['2.2', '2.3'], 'networks': {'ovirtmgmt': {'iface': 'ovirtmgmt', 'addr': '10.35.0.100', 'cfg': {'DEFROUTE': 'yes', 'HOTPLUG': 'no', 'MTU': '1500', 'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'STP': 'off', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'bridged': True, 'ipv6addrs': ['fe80::5054:ff:fe87:196f/64'], 'gateway': '10.35.1.254', 'bootproto4': 'dhcp', 'netmask': '255.255.254.0', 'stp': 'off', 'ipv4addrs': ['10.35.0.100/23'], 'mtu': '1500', 'ipv6gateway': '::', 'ports': ['eth0']}}, 'bridges': {'ovirtmgmt': {'addr': '10.35.0.100', 'cfg': {'DEFROUTE': 'yes', 'HOTPLUG': 'no', 'MTU': '1500', 'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'STP': 'off', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'ipv6addrs': ['fe80::5054:ff:fe87:196f/64'], 'mtu': '1500', 'netmask': '255.255.254.0', 'stp': 'off', 'ipv4addrs': ['10.35.0.100/23'], 'ipv6gateway': '::', 'gateway': '10.35.1.254', 'opts': {'topology_change_detected': '0', 'multicast_last_member_count': '2', 'hash_elasticity': '4', 'multicast_query_response_interval': '999', 'multicast_snooping': '1', 'multicast_startup_query_interval': '3124', 'hello_timer': '141', 'multicast_querier_interval': '25496', 'max_age': '1999', 'hash_max': '512', 'stp_state': '0', 'root_id': '8000.52540087196f', 'priority': '32768', 'multicast_membership_interval': '25996', 'root_path_cost': '0', 'root_port': '0', 'multicast_querier': '0', 'multicast_startup_query_count': '2', 'hello_time': '199', 'topology_change': '0', 'bridge_id': '8000.52540087196f', 'topology_change_timer': '0', 'ageing_time': '29995', 'gc_timer': '41', 'group_addr': '1:80:c2:0:0:0', 'tcn_timer': '0', 'multicast_query_interval': '12498', 'multicast_last_member_interval': '99', 'multicast_router': '1', 'forward_delay': '0'}, 'ports': ['eth0']}}, 'uuid': '85CFDED0-FD24-4645-AF7C-D47A088AE9E8', 'lastClientIface': 'ovirtmgmt', 'nics': {'eth0': {'addr': '', 'cfg': {'BRIDGE': 'ovirtmgmt', 'NM_CONTROLLED': 'no', 'MTU': '1500', 'HWADDR': '52:54:00:87:19:6f', 'DEVICE': 'eth0', 'ONBOOT': 'yes'}, 'ipv6addrs': [], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': '52:54:00:87:19:6f', 'speed': 0}}, 'software_revision': '0', 'clusterLevels': ['3.0', '3.1', '3.2', '3.3', '3.4', '3.5'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,mmx,fxsr,sse,sse2,ss,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,rep_good,unfair_spinlock,pni,pclmulqdq,vmx,ssse3,fma,cx16,pcid,sse4_1,sse4_2,x2apic,movbe,popcnt,tsc_deadline_timer,aes,xsave,avx,f16c,rdrand,hypervisor,lahf_lm,abm,xsaveopt,vnmi,ept,fsgsbase,bmi1,hle,avx2,smep,bmi2,erms,invpcid,rtm,model_Haswell,model_Nehalem,model_Conroe,model_Penryn,model_Westmere,model_SandyBridge', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:nsoffer1', 'netConfigDirty': 'False', 'supportedENGINEs': ['3.0', '3.1', '3.2', '3.3', '3.4', '3.5'], 'autoNumaBalancing': 2, 'reservedMem': '321', 'bondings': {'bond4': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond0': {'addr': '', 'cfg': {}, 'ipv6addrs': [], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': '00:00:00:00:00:00', 'slaves': [], 'opts': {}}, 'bond1': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond2': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond3': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}}, 'software_version': '4.16', 'memSize': '3734', 'cpuSpeed': '3392.160', 'numaNodes': {u'0': {'totalMemory': '3734', 'cpus': [0, 1, 2, 3, 4, 5, 6, 7]}}, 'version_name': 'Snow Man', 'vlans': {}, 'cpuCores': '8', 'kvmEnabled': 'true', 'guestOverhead': '65', 'cpuThreads': '8', 'emulatedMachines': [u'rhel6.5.0', u'pc', u'rhel6.4.0', u'rhel6.3.0', u'rhel6.2.0', u'rhel6.1.0', u'rhel6.0.0', u'rhel5.5.0', u'rhel5.4.4', u'rhel5.4.0'], 'operatingSystem': {'release': '6.5.0.1.el6', 'version': '6Server', 'name': 'RHEL'}} >Thread-16::DEBUG::2014-10-05 21:30:02,926::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:30:03,111::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:30:03,112::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-17::DEBUG::2014-10-05 21:30:03,112::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getHardwareInfo' in bridge with {} >Thread-17::DEBUG::2014-10-05 21:30:03,152::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.getHardwareInfo' in bridge with {'systemProductName': 'Bochs', 'systemUUID': '85cfded0-fd24-4645-af7c-d47a088ae9e8', 'systemManufacturer': 'Bochs'} >Thread-17::DEBUG::2014-10-05 21:30:03,152::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:30:03,366::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:30:03,367::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-18::DEBUG::2014-10-05 21:30:03,367::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:30:03,551::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-19::DEBUG::2014-10-05 21:30:03,552::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.connectStorageServer' in bridge with {u'connectionParams': [{u'id': u'af150e8e-4d74-4810-b8bf-0c6a19637dbf', u'connection': u'10.35.16.25', u'iqn': u'iqn.1994-05.com.redhat:nsoffer-target1', u'user': u'', u'tpgt': u'1', u'password': u'', u'port': u'3260'}, {u'id': u'e8d58651-6414-4fc2-b281-2141cc97829b', u'connection': u'10.35.0.98', u'iqn': u'iqn.1994-05.com.redhat:target6', u'user': u'', u'tpgt': u'1', u'password': u'', u'port': u'3260'}, {u'id': u'8f399b5f-1237-479e-82e6-eedb202b66fd', u'connection': u'10.35.16.25', u'iqn': u'iqn.1994-05.com.redhat:nsoffer-target2', u'user': u'', u'tpgt': u'1', u'password': u'', u'port': u'3260'}], u'storagepoolID': u'b86b687a-d073-497a-ac8a-249025419a3e', u'domainType': 3} >JsonRpcServer::DEBUG::2014-10-05 21:30:03,552::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-19::DEBUG::2014-10-05 21:30:03,553::task::595::Storage.TaskManager.Task::(_updateState) Task=`ba99a4e6-bcad-4d2f-b023-98dcedf02044`::moving from state init -> state preparing >Thread-19::INFO::2014-10-05 21:30:03,553::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=3, spUUID=u'b86b687a-d073-497a-ac8a-249025419a3e', conList=[{u'connection': u'10.35.16.25', u'iqn': u'iqn.1994-05.com.redhat:nsoffer-target1', u'user': u'', u'tpgt': u'1', u'password': '******', u'id': u'af150e8e-4d74-4810-b8bf-0c6a19637dbf', u'port': u'3260'}, {u'connection': u'10.35.0.98', u'iqn': u'iqn.1994-05.com.redhat:target6', u'user': u'', u'tpgt': u'1', u'password': '******', u'id': u'e8d58651-6414-4fc2-b281-2141cc97829b', u'port': u'3260'}, {u'connection': u'10.35.16.25', u'iqn': u'iqn.1994-05.com.redhat:nsoffer-target2', u'user': u'', u'tpgt': u'1', u'password': '******', u'id': u'8f399b5f-1237-479e-82e6-eedb202b66fd', u'port': u'3260'}], options=None) >Thread-19::DEBUG::2014-10-05 21:30:03,554::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:nsoffer-target1 -I default -p 10.35.16.25:3260,1 --op=new (cwd None) >Thread-19::DEBUG::2014-10-05 21:30:03,564::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-19::DEBUG::2014-10-05 21:30:03,565::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:nsoffer-target1 -I default -p 10.35.16.25:3260,1 -l (cwd None) >Thread-19::DEBUG::2014-10-05 21:30:04,350::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-19::DEBUG::2014-10-05 21:30:04,351::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:nsoffer-target1 -I default -p 10.35.16.25:3260,1 -n node.startup -v manual --op=update (cwd None) >Thread-19::DEBUG::2014-10-05 21:30:04,361::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-19::DEBUG::2014-10-05 21:30:04,361::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-19::DEBUG::2014-10-05 21:30:04,362::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-19::DEBUG::2014-10-05 21:30:04,465::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-19::DEBUG::2014-10-05 21:30:04,465::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-19::DEBUG::2014-10-05 21:30:04,465::hsm::2396::Storage.HSM::(__prefetchDomains) Found SD uuids: () >Thread-19::DEBUG::2014-10-05 21:30:04,465::hsm::2452::Storage.HSM::(connectStorageServer) knownSDs: {} >Thread-19::DEBUG::2014-10-05 21:30:04,466::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:target6 -I default -p 10.35.0.98:3260,1 --op=new (cwd None) >Thread-19::DEBUG::2014-10-05 21:30:04,482::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-19::DEBUG::2014-10-05 21:30:04,482::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:target6 -I default -p 10.35.0.98:3260,1 -l (cwd None) >Thread-19::DEBUG::2014-10-05 21:30:05,077::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-19::DEBUG::2014-10-05 21:30:05,078::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:target6 -I default -p 10.35.0.98:3260,1 -n node.startup -v manual --op=update (cwd None) >Thread-19::DEBUG::2014-10-05 21:30:05,100::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-19::DEBUG::2014-10-05 21:30:05,100::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-19::DEBUG::2014-10-05 21:30:05,101::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-19::DEBUG::2014-10-05 21:30:05,265::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-19::DEBUG::2014-10-05 21:30:05,266::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-19::DEBUG::2014-10-05 21:30:05,266::hsm::2396::Storage.HSM::(__prefetchDomains) Found SD uuids: () >Thread-19::DEBUG::2014-10-05 21:30:05,267::hsm::2452::Storage.HSM::(connectStorageServer) knownSDs: {} >Thread-19::DEBUG::2014-10-05 21:30:05,267::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:nsoffer-target2 -I default -p 10.35.16.25:3260,1 --op=new (cwd None) >Thread-19::DEBUG::2014-10-05 21:30:05,281::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-19::DEBUG::2014-10-05 21:30:05,281::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:nsoffer-target2 -I default -p 10.35.16.25:3260,1 -l (cwd None) >Thread-19::DEBUG::2014-10-05 21:30:06,094::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-19::DEBUG::2014-10-05 21:30:06,094::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:nsoffer-target2 -I default -p 10.35.16.25:3260,1 -n node.startup -v manual --op=update (cwd None) >Thread-19::DEBUG::2014-10-05 21:30:06,106::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-19::DEBUG::2014-10-05 21:30:06,106::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-19::DEBUG::2014-10-05 21:30:06,106::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-19::DEBUG::2014-10-05 21:30:06,181::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-19::DEBUG::2014-10-05 21:30:06,182::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-19::DEBUG::2014-10-05 21:30:06,182::hsm::2396::Storage.HSM::(__prefetchDomains) Found SD uuids: () >Thread-19::DEBUG::2014-10-05 21:30:06,182::hsm::2452::Storage.HSM::(connectStorageServer) knownSDs: {} >Thread-19::INFO::2014-10-05 21:30:06,182::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': u'af150e8e-4d74-4810-b8bf-0c6a19637dbf'}, {'status': 0, 'id': u'e8d58651-6414-4fc2-b281-2141cc97829b'}, {'status': 0, 'id': u'8f399b5f-1237-479e-82e6-eedb202b66fd'}]} >Thread-19::DEBUG::2014-10-05 21:30:06,187::task::1191::Storage.TaskManager.Task::(prepare) Task=`ba99a4e6-bcad-4d2f-b023-98dcedf02044`::finished: {'statuslist': [{'status': 0, 'id': u'af150e8e-4d74-4810-b8bf-0c6a19637dbf'}, {'status': 0, 'id': u'e8d58651-6414-4fc2-b281-2141cc97829b'}, {'status': 0, 'id': u'8f399b5f-1237-479e-82e6-eedb202b66fd'}]} >Thread-19::DEBUG::2014-10-05 21:30:06,187::task::595::Storage.TaskManager.Task::(_updateState) Task=`ba99a4e6-bcad-4d2f-b023-98dcedf02044`::moving from state preparing -> state finished >Thread-19::DEBUG::2014-10-05 21:30:06,187::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-19::DEBUG::2014-10-05 21:30:06,187::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-19::DEBUG::2014-10-05 21:30:06,187::task::993::Storage.TaskManager.Task::(_decref) Task=`ba99a4e6-bcad-4d2f-b023-98dcedf02044`::ref 0 aborting False >Thread-19::DEBUG::2014-10-05 21:30:06,188::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.connectStorageServer' in bridge with [{'status': 0, 'id': u'af150e8e-4d74-4810-b8bf-0c6a19637dbf'}, {'status': 0, 'id': u'e8d58651-6414-4fc2-b281-2141cc97829b'}, {'status': 0, 'id': u'8f399b5f-1237-479e-82e6-eedb202b66fd'}] >Thread-19::DEBUG::2014-10-05 21:30:06,188::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:30:06,375::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-20::DEBUG::2014-10-05 21:30:06,375::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.connect' in bridge with {u'masterVersion': 5, u'domainDict': {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': u'active', u'ff559f46-c495-4f6b-901c-2a624042a050': u'active'}, u'storagepoolID': u'b86b687a-d073-497a-ac8a-249025419a3e', u'scsiKey': u'b86b687a-d073-497a-ac8a-249025419a3e', u'masterSdUUID': u'ff559f46-c495-4f6b-901c-2a624042a050', u'hostID': 3} >JsonRpcServer::DEBUG::2014-10-05 21:30:06,376::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-20::DEBUG::2014-10-05 21:30:06,377::task::595::Storage.TaskManager.Task::(_updateState) Task=`2b318ae5-5ba9-4d19-afb0-75ac53b14a13`::moving from state init -> state preparing >Thread-20::INFO::2014-10-05 21:30:06,377::logUtils::44::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID=u'b86b687a-d073-497a-ac8a-249025419a3e', hostID=3, msdUUID=u'ff559f46-c495-4f6b-901c-2a624042a050', masterVersion=5, domainsMap={u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': u'active', u'ff559f46-c495-4f6b-901c-2a624042a050': u'active'}, options=None) >Thread-20::DEBUG::2014-10-05 21:30:06,377::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.HsmDomainMonitorLock`ReqID=`39ef896f-755a-4354-a74b-6340899c2a01`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1011' at 'connectStoragePool' >Thread-20::DEBUG::2014-10-05 21:30:06,377::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.HsmDomainMonitorLock' for lock type 'exclusive' >Thread-20::DEBUG::2014-10-05 21:30:06,377::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.HsmDomainMonitorLock' is free. Now locking as 'exclusive' (1 active user) >Thread-20::DEBUG::2014-10-05 21:30:06,377::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.HsmDomainMonitorLock`ReqID=`39ef896f-755a-4354-a74b-6340899c2a01`::Granted request >Thread-20::DEBUG::2014-10-05 21:30:06,378::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.b86b687a-d073-497a-ac8a-249025419a3e`ReqID=`db1807c1-2405-4633-ba22-e4c29eadbdfc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1059' at '_connectStoragePool' >Thread-20::DEBUG::2014-10-05 21:30:06,378::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.b86b687a-d073-497a-ac8a-249025419a3e' for lock type 'exclusive' >Thread-20::DEBUG::2014-10-05 21:30:06,378::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.b86b687a-d073-497a-ac8a-249025419a3e' is free. Now locking as 'exclusive' (1 active user) >Thread-20::DEBUG::2014-10-05 21:30:06,378::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.b86b687a-d073-497a-ac8a-249025419a3e`ReqID=`db1807c1-2405-4633-ba22-e4c29eadbdfc`::Granted request >Thread-20::DEBUG::2014-10-05 21:30:06,378::spbackends::485::Storage.StoragePoolMemoryBackend::(updateVersionAndDomains) updating domain version to 5 and domains map to {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': u'active', u'ff559f46-c495-4f6b-901c-2a624042a050': u'active'} >Thread-20::INFO::2014-10-05 21:30:06,378::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 5 and domains map {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': u'Active', u'ff559f46-c495-4f6b-901c-2a624042a050': u'Active'} >Thread-20::INFO::2014-10-05 21:30:06,378::sp::133::Storage.StoragePool::(setBackend) updating pool b86b687a-d073-497a-ac8a-249025419a3e backend from type NoneType instance 0x3a11372210 to type StoragePoolMemoryBackend instance 0x7f6fd4243730 >Thread-20::INFO::2014-10-05 21:30:06,379::sp::631::Storage.StoragePool::(connect) Connect host #3 to the storage pool b86b687a-d073-497a-ac8a-249025419a3e with master domain: ff559f46-c495-4f6b-901c-2a624042a050 (ver = 5) >Thread-20::DEBUG::2014-10-05 21:30:06,379::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:06,379::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:06,379::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:06,379::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:06,379::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:06,379::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:06,379::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-20::DEBUG::2014-10-05 21:30:06,379::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-20::DEBUG::2014-10-05 21:30:06,379::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-20::DEBUG::2014-10-05 21:30:06,379::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-20::DEBUG::2014-10-05 21:30:06,380::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-20::DEBUG::2014-10-05 21:30:06,380::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-20::DEBUG::2014-10-05 21:30:06,709::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-20::DEBUG::2014-10-05 21:30:06,730::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-20::DEBUG::2014-10-05 21:30:06,934::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-20::DEBUG::2014-10-05 21:30:06,935::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:06,935::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:06,935::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:06,935::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:06,935::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:06,935::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:06,936::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-20::ERROR::2014-10-05 21:30:06,936::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain ff559f46-c495-4f6b-901c-2a624042a050 >Thread-20::ERROR::2014-10-05 21:30:06,936::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain ff559f46-c495-4f6b-901c-2a624042a050 >Thread-20::DEBUG::2014-10-05 21:30:06,936::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:06,939::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060001|/dev/mapper/1IET_00060002|/dev/mapper/1IET_00060003|/dev/mapper/1IET_00060004|/dev/mapper/1IET_00060005|/dev/mapper/1IET_00060006|/dev/mapper/1IET_00060007|/dev/mapper/1IET_00060008|/dev/mapper/1IET_00060009|/dev/mapper/1IET_0006000a|/dev/mapper/3600144f09dbd05000000522dde040011|/dev/mapper/3600144f09dbd05000000522dde050012|/dev/mapper/3600144f09dbd05000000522dde070013|/dev/mapper/3600144f09dbd05000000522dde090014|/dev/mapper/3600144f09dbd05000000522dde0a0015|/dev/mapper/3600144f09dbd05000000522dde0c0016|/dev/mapper/3600144f09dbd05000000522dde0d0017|/dev/mapper/3600144f09dbd05000000522dde0f0018|/dev/mapper/3600144f09dbd05000000522dde100019|/dev/mapper/3600144f09dbd05000000522dde12001a|/dev/mapper/3600144f09dbd0500000053a84cba0002|/dev/mapper/3600144f09dbd0500000053a84e730003|/dev/mapper/3600144f09dbd0500000053a84f4d0004|/dev/mapper/3600144f09dbd0500000053a8503b0005|/dev/mapper/3600144f09dbd0500000053a850500006|/dev/mapper/3600144f09dbd0500000053a8505b0007|/dev/mapper/3600144f09dbd0500000053a850640008|/dev/mapper/3600144f09dbd0500000053a8506c0009|/dev/mapper/3600144f09dbd0500000053a85076000a|/dev/mapper/3600144f09dbd0500000053a8507f000b|/dev/mapper/3600144f09dbd0500000053a853ed000c|/dev/mapper/3600144f09dbd0500000053a85401000d|/dev/mapper/3600144f09dbd0500000053a85409000e|/dev/mapper/3600144f09dbd0500000053a85410000f|/dev/mapper/3600144f09dbd0500000053a854180010|/dev/mapper/3600144f09dbd0500000053a854250011|/dev/mapper/3600144f09dbd0500000053a8542d0012|/dev/mapper/3600144f09dbd0500000053a8543e0013|/dev/mapper/3600144f09dbd0500000053a854450014|/dev/mapper/3600144f09dbd0500000053a8544d0015|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name ff559f46-c495-4f6b-901c-2a624042a050 (cwd None) >Thread-20::DEBUG::2014-10-05 21:30:07,165::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-20::DEBUG::2014-10-05 21:30:07,165::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:07,165::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-20::DEBUG::2014-10-05 21:30:07,166::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060007|/dev/mapper/1IET_00060009|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --available y ff559f46-c495-4f6b-901c-2a624042a050/metadata (cwd None) >Thread-20::DEBUG::2014-10-05 21:30:07,302::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-20::DEBUG::2014-10-05 21:30:07,302::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:07,302::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:07,302::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata count=1 (cwd None) >Thread-20::DEBUG::2014-10-05 21:30:07,307::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000678452 s, 3.0 MB/s\n'; <rc> = 0 >Thread-20::DEBUG::2014-10-05 21:30:07,307::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000678452 s, 3.0 MB/s'], size: 2048 >Thread-20::DEBUG::2014-10-05 21:30:07,307::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-20::DEBUG::2014-10-05 21:30:07,307::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-20::DEBUG::2014-10-05 21:30:07,307::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-20::DEBUG::2014-10-05 21:30:07,308::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:07,308::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:07,308::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:07,308::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:07,308::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:07,308::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060001|/dev/mapper/1IET_00060002|/dev/mapper/1IET_00060003|/dev/mapper/1IET_00060004|/dev/mapper/1IET_00060005|/dev/mapper/1IET_00060006|/dev/mapper/1IET_00060007|/dev/mapper/1IET_00060008|/dev/mapper/1IET_00060009|/dev/mapper/1IET_0006000a|/dev/mapper/3600144f09dbd05000000522dde040011|/dev/mapper/3600144f09dbd05000000522dde050012|/dev/mapper/3600144f09dbd05000000522dde070013|/dev/mapper/3600144f09dbd05000000522dde090014|/dev/mapper/3600144f09dbd05000000522dde0a0015|/dev/mapper/3600144f09dbd05000000522dde0c0016|/dev/mapper/3600144f09dbd05000000522dde0d0017|/dev/mapper/3600144f09dbd05000000522dde0f0018|/dev/mapper/3600144f09dbd05000000522dde100019|/dev/mapper/3600144f09dbd05000000522dde12001a|/dev/mapper/3600144f09dbd0500000053a84cba0002|/dev/mapper/3600144f09dbd0500000053a84e730003|/dev/mapper/3600144f09dbd0500000053a84f4d0004|/dev/mapper/3600144f09dbd0500000053a8503b0005|/dev/mapper/3600144f09dbd0500000053a850500006|/dev/mapper/3600144f09dbd0500000053a8505b0007|/dev/mapper/3600144f09dbd0500000053a850640008|/dev/mapper/3600144f09dbd0500000053a8506c0009|/dev/mapper/3600144f09dbd0500000053a85076000a|/dev/mapper/3600144f09dbd0500000053a8507f000b|/dev/mapper/3600144f09dbd0500000053a853ed000c|/dev/mapper/3600144f09dbd0500000053a85401000d|/dev/mapper/3600144f09dbd0500000053a85409000e|/dev/mapper/3600144f09dbd0500000053a85410000f|/dev/mapper/3600144f09dbd0500000053a854180010|/dev/mapper/3600144f09dbd0500000053a854250011|/dev/mapper/3600144f09dbd0500000053a8542d0012|/dev/mapper/3600144f09dbd0500000053a8543e0013|/dev/mapper/3600144f09dbd0500000053a854450014|/dev/mapper/3600144f09dbd0500000053a8544d0015|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name ff559f46-c495-4f6b-901c-2a624042a050 (cwd None) >Thread-20::DEBUG::2014-10-05 21:30:07,530::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-20::DEBUG::2014-10-05 21:30:07,530::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:07,531::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'VGUUID=1B1oT5-JcGj-Ru1I-v3Ni-yden-Du9b-I9fSu8', 'CLASS=Data', 'VERSION=3', 'TYPE=ISCSI', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'LOCKPOLICY=', 'SDUUID=ff559f46-c495-4f6b-901c-2a624042a050', 'PHYBLKSIZE=512', u'PV0=pv:1IET_00060007,uuid:QfuzQ5-Ml7u-kFUc-gdmr-rfmS-Aefp-3HODNb,pestart:0,pecount:157,mapoffset:0', 'POOL_UUID=b86b687a-d073-497a-ac8a-249025419a3e', 'ROLE=Master', u'PV1=pv:1IET_00060009,uuid:pGRciD-wcLp-X1ME-MijE-P1B9-7XWo-ycTXdG,pestart:0,pecount:157,mapoffset:157', 'DESCRIPTION=jumbo-el7', '_SHA_CKSUM=1695e16fba6531db32e52743127e50d763522d29'] >Thread-20::DEBUG::2014-10-05 21:30:07,531::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060007|/dev/mapper/1IET_00060009|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --available y ff559f46-c495-4f6b-901c-2a624042a050/ids (cwd None) >Thread-20::DEBUG::2014-10-05 21:30:07,671::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-20::DEBUG::2014-10-05 21:30:07,671::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:07,672::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:07,672::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060007|/dev/mapper/1IET_00060009|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --available y ff559f46-c495-4f6b-901c-2a624042a050/leases (cwd None) >Thread-20::DEBUG::2014-10-05 21:30:07,811::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-20::DEBUG::2014-10-05 21:30:07,811::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:07,812::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:07,812::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060007|/dev/mapper/1IET_00060009|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --available y ff559f46-c495-4f6b-901c-2a624042a050/inbox ff559f46-c495-4f6b-901c-2a624042a050/outbox ff559f46-c495-4f6b-901c-2a624042a050/master (cwd None) >Thread-20::DEBUG::2014-10-05 21:30:07,973::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-20::DEBUG::2014-10-05 21:30:07,973::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:07,973::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:07,974::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'ff559f46-c495-4f6b-901c-2a624042a050_imageNS' >Thread-20::DEBUG::2014-10-05 21:30:07,974::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'ff559f46-c495-4f6b-901c-2a624042a050_volumeNS' >Thread-20::DEBUG::2014-10-05 21:30:07,974::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'ff559f46-c495-4f6b-901c-2a624042a050_lvmActivationNS' >Thread-20::DEBUG::2014-10-05 21:30:07,974::sp::1390::Storage.StoragePool::(setMasterDomain) Master domain ff559f46-c495-4f6b-901c-2a624042a050 verified, version 5 >Thread-20::DEBUG::2014-10-05 21:30:07,974::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-20::DEBUG::2014-10-05 21:30:07,975::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-20::INFO::2014-10-05 21:30:07,975::domainMonitor::100::Storage.DomainMonitor::(startMonitoring) Start monitoring c73a3ced-4b41-4b4e-93f7-494d63e7e7ae >Thread-21::DEBUG::2014-10-05 21:30:07,975::domainMonitor::182::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for c73a3ced-4b41-4b4e-93f7-494d63e7e7ae >Thread-20::DEBUG::2014-10-05 21:30:07,975::sp::1413::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `b86b687a-d073-497a-ac8a-249025419a3e` started monitoring domain `c73a3ced-4b41-4b4e-93f7-494d63e7e7ae` >Thread-21::ERROR::2014-10-05 21:30:07,976::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain c73a3ced-4b41-4b4e-93f7-494d63e7e7ae >Thread-20::INFO::2014-10-05 21:30:07,976::domainMonitor::100::Storage.DomainMonitor::(startMonitoring) Start monitoring ff559f46-c495-4f6b-901c-2a624042a050 >Thread-21::ERROR::2014-10-05 21:30:07,976::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain c73a3ced-4b41-4b4e-93f7-494d63e7e7ae >Thread-21::DEBUG::2014-10-05 21:30:07,977::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-21::DEBUG::2014-10-05 21:30:07,977::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060001|/dev/mapper/1IET_00060002|/dev/mapper/1IET_00060003|/dev/mapper/1IET_00060004|/dev/mapper/1IET_00060005|/dev/mapper/1IET_00060006|/dev/mapper/1IET_00060007|/dev/mapper/1IET_00060008|/dev/mapper/1IET_00060009|/dev/mapper/1IET_0006000a|/dev/mapper/3600144f09dbd05000000522dde040011|/dev/mapper/3600144f09dbd05000000522dde050012|/dev/mapper/3600144f09dbd05000000522dde070013|/dev/mapper/3600144f09dbd05000000522dde090014|/dev/mapper/3600144f09dbd05000000522dde0a0015|/dev/mapper/3600144f09dbd05000000522dde0c0016|/dev/mapper/3600144f09dbd05000000522dde0d0017|/dev/mapper/3600144f09dbd05000000522dde0f0018|/dev/mapper/3600144f09dbd05000000522dde100019|/dev/mapper/3600144f09dbd05000000522dde12001a|/dev/mapper/3600144f09dbd0500000053a84cba0002|/dev/mapper/3600144f09dbd0500000053a84e730003|/dev/mapper/3600144f09dbd0500000053a84f4d0004|/dev/mapper/3600144f09dbd0500000053a8503b0005|/dev/mapper/3600144f09dbd0500000053a850500006|/dev/mapper/3600144f09dbd0500000053a8505b0007|/dev/mapper/3600144f09dbd0500000053a850640008|/dev/mapper/3600144f09dbd0500000053a8506c0009|/dev/mapper/3600144f09dbd0500000053a85076000a|/dev/mapper/3600144f09dbd0500000053a8507f000b|/dev/mapper/3600144f09dbd0500000053a853ed000c|/dev/mapper/3600144f09dbd0500000053a85401000d|/dev/mapper/3600144f09dbd0500000053a85409000e|/dev/mapper/3600144f09dbd0500000053a85410000f|/dev/mapper/3600144f09dbd0500000053a854180010|/dev/mapper/3600144f09dbd0500000053a854250011|/dev/mapper/3600144f09dbd0500000053a8542d0012|/dev/mapper/3600144f09dbd0500000053a8543e0013|/dev/mapper/3600144f09dbd0500000053a854450014|/dev/mapper/3600144f09dbd0500000053a8544d0015|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name c73a3ced-4b41-4b4e-93f7-494d63e7e7ae (cwd None) >Thread-20::DEBUG::2014-10-05 21:30:07,984::sp::1413::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `b86b687a-d073-497a-ac8a-249025419a3e` started monitoring domain `ff559f46-c495-4f6b-901c-2a624042a050` >Thread-20::DEBUG::2014-10-05 21:30:07,986::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-22::DEBUG::2014-10-05 21:30:07,986::domainMonitor::182::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for ff559f46-c495-4f6b-901c-2a624042a050 >Thread-20::DEBUG::2014-10-05 21:30:07,986::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e >Thread-22::DEBUG::2014-10-05 21:30:07,988::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgck --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060007|/dev/mapper/1IET_00060009|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' ff559f46-c495-4f6b-901c-2a624042a050 (cwd None) >Thread-20::DEBUG::2014-10-05 21:30:07,996::lvm::365::Storage.OperationMutex::(_reloadvgs) Got the operational mutex >Thread-20::DEBUG::2014-10-05 21:30:07,996::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060001|/dev/mapper/1IET_00060002|/dev/mapper/1IET_00060003|/dev/mapper/1IET_00060004|/dev/mapper/1IET_00060005|/dev/mapper/1IET_00060006|/dev/mapper/1IET_00060007|/dev/mapper/1IET_00060008|/dev/mapper/1IET_00060009|/dev/mapper/1IET_0006000a|/dev/mapper/3600144f09dbd05000000522dde040011|/dev/mapper/3600144f09dbd05000000522dde050012|/dev/mapper/3600144f09dbd05000000522dde070013|/dev/mapper/3600144f09dbd05000000522dde090014|/dev/mapper/3600144f09dbd05000000522dde0a0015|/dev/mapper/3600144f09dbd05000000522dde0c0016|/dev/mapper/3600144f09dbd05000000522dde0d0017|/dev/mapper/3600144f09dbd05000000522dde0f0018|/dev/mapper/3600144f09dbd05000000522dde100019|/dev/mapper/3600144f09dbd05000000522dde12001a|/dev/mapper/3600144f09dbd0500000053a84cba0002|/dev/mapper/3600144f09dbd0500000053a84e730003|/dev/mapper/3600144f09dbd0500000053a84f4d0004|/dev/mapper/3600144f09dbd0500000053a8503b0005|/dev/mapper/3600144f09dbd0500000053a850500006|/dev/mapper/3600144f09dbd0500000053a8505b0007|/dev/mapper/3600144f09dbd0500000053a850640008|/dev/mapper/3600144f09dbd0500000053a8506c0009|/dev/mapper/3600144f09dbd0500000053a85076000a|/dev/mapper/3600144f09dbd0500000053a8507f000b|/dev/mapper/3600144f09dbd0500000053a853ed000c|/dev/mapper/3600144f09dbd0500000053a85401000d|/dev/mapper/3600144f09dbd0500000053a85409000e|/dev/mapper/3600144f09dbd0500000053a85410000f|/dev/mapper/3600144f09dbd0500000053a854180010|/dev/mapper/3600144f09dbd0500000053a854250011|/dev/mapper/3600144f09dbd0500000053a8542d0012|/dev/mapper/3600144f09dbd0500000053a8543e0013|/dev/mapper/3600144f09dbd0500000053a854450014|/dev/mapper/3600144f09dbd0500000053a8544d0015|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name c73a3ced-4b41-4b4e-93f7-494d63e7e7ae (cwd None) >Thread-22::DEBUG::2014-10-05 21:30:08,145::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-22::DEBUG::2014-10-05 21:30:08,145::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-10-05 21:30:08,149::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000451325 s, 9.1 MB/s\n'; <rc> = 0 >Thread-22::DEBUG::2014-10-05 21:30:08,153::domainMonitor::264::Storage.DomainMonitorThread::(_monitorDomain) Domain ff559f46-c495-4f6b-901c-2a624042a050 changed its status to Valid >Thread-22::INFO::2014-10-05 21:30:08,153::clusterlock::222::Storage.SANLock::(acquireHostId) Acquiring host id for domain ff559f46-c495-4f6b-901c-2a624042a050 (id: 3) >Thread-23::DEBUG::2014-10-05 21:30:08,153::misc::821::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Emitting event >Thread-23::DEBUG::2014-10-05 21:30:08,154::misc::831::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `contEIOVms` >Thread-23::DEBUG::2014-10-05 21:30:08,154::misc::841::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Event emitted >Thread-22::DEBUG::2014-10-05 21:30:08,154::clusterlock::240::Storage.SANLock::(acquireHostId) Host id for domain ff559f46-c495-4f6b-901c-2a624042a050 successfully acquired (id: 3) >Thread-24::INFO::2014-10-05 21:30:08,155::clientIF::136::vds::(contEIOVms) vmContainerLock acquired >Thread-21::DEBUG::2014-10-05 21:30:08,208::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-21::DEBUG::2014-10-05 21:30:08,208::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-21::DEBUG::2014-10-05 21:30:08,209::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/3600144f09dbd0500000053a84e730003|/dev/mapper/3600144f09dbd0500000053a85076000a|/dev/mapper/3600144f09dbd0500000053a853ed000c|/dev/mapper/3600144f09dbd0500000053a85401000d|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --available y c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata (cwd None) >Thread-20::DEBUG::2014-10-05 21:30:08,284::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-20::DEBUG::2014-10-05 21:30:08,284::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-20::DEBUG::2014-10-05 21:30:08,284::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/dom_md >Thread-20::WARNING::2014-10-05 21:30:08,285::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/dom_md already exists >Thread-20::DEBUG::2014-10-05 21:30:08,285::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/images >Thread-20::WARNING::2014-10-05 21:30:08,285::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/images already exists >Thread-20::INFO::2014-10-05 21:30:08,285::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae to /rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae >Thread-20::DEBUG::2014-10-05 21:30:08,285::sp::1137::Storage.StoragePool::(_linkStorageDomain) Creating symlink from /rhev/data-center/mnt/blockSD/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae to /rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae >Thread-20::DEBUG::2014-10-05 21:30:08,286::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050/images >Thread-20::WARNING::2014-10-05 21:30:08,286::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050/images already exists >Thread-20::DEBUG::2014-10-05 21:30:08,286::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050/dom_md >Thread-20::WARNING::2014-10-05 21:30:08,286::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050/dom_md already exists >Thread-20::INFO::2014-10-05 21:30:08,286::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050 to /rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/ff559f46-c495-4f6b-901c-2a624042a050 >Thread-20::DEBUG::2014-10-05 21:30:08,286::sp::1137::Storage.StoragePool::(_linkStorageDomain) Creating symlink from /rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050 to /rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/ff559f46-c495-4f6b-901c-2a624042a050 >Thread-20::INFO::2014-10-05 21:30:08,286::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050 to /rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/mastersd >Thread-20::DEBUG::2014-10-05 21:30:08,287::sp::1137::Storage.StoragePool::(_linkStorageDomain) Creating symlink from /rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050 to /rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/mastersd >Thread-20::DEBUG::2014-10-05 21:30:08,287::threadPool::35::Storage.ThreadPool::(__init__) Enter - numThreads: 5, waitTimeout: 3, maxTasks: 500 >Thread-20::DEBUG::2014-10-05 21:30:08,290::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /bin/dd if=/rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/mastersd/dom_md/outbox iflag=direct,fullblock bs=512 count=8 skip=24 (cwd None) >Thread-20::DEBUG::2014-10-05 21:30:08,297::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.00358575 s, 1.1 MB/s\n'; <rc> = 0 >Thread-20::INFO::2014-10-05 21:30:08,297::storage_mailbox::385::Storage.MailBox.HsmMailMonitor::(_sendMail) HSM_MailMonitor sending mail to SPM - ['/bin/dd', 'of=/rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/mastersd/dom_md/inbox', 'iflag=fullblock', 'oflag=direct', 'conv=notrunc', 'bs=512', 'seek=24'] >Thread-20::DEBUG::2014-10-05 21:30:08,298::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /bin/dd of=/rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/mastersd/dom_md/inbox iflag=fullblock oflag=direct conv=notrunc bs=512 seek=24 (cwd None) >Thread-20::DEBUG::2014-10-05 21:30:08,305::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.0030293 s, 1.4 MB/s\n'; <rc> = 0 >Thread-20::DEBUG::2014-10-05 21:30:08,306::storage_mailbox::207::Storage.Mailbox.HSM::(__init__) HSM_MailboxMonitor created for pool b86b687a-d073-497a-ac8a-249025419a3e >Thread-20::DEBUG::2014-10-05 21:30:08,306::sp::462::Storage.StoragePool::(__createMailboxMonitor) HSM mailbox ready for pool b86b687a-d073-497a-ac8a-249025419a3e on master domain ff559f46-c495-4f6b-901c-2a624042a050 >Thread-20::DEBUG::2014-10-05 21:30:08,306::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.b86b687a-d073-497a-ac8a-249025419a3e' >Thread-20::DEBUG::2014-10-05 21:30:08,306::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.b86b687a-d073-497a-ac8a-249025419a3e' (0 active users) >Thread-20::DEBUG::2014-10-05 21:30:08,307::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.b86b687a-d073-497a-ac8a-249025419a3e' is free, finding out if anyone is waiting for it. >Thread-20::DEBUG::2014-10-05 21:30:08,307::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.b86b687a-d073-497a-ac8a-249025419a3e', Clearing records. >Thread-20::DEBUG::2014-10-05 21:30:08,307::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.HsmDomainMonitorLock' >Thread-20::DEBUG::2014-10-05 21:30:08,307::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.HsmDomainMonitorLock' (0 active users) >Thread-20::DEBUG::2014-10-05 21:30:08,307::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.HsmDomainMonitorLock' is free, finding out if anyone is waiting for it. >Thread-20::DEBUG::2014-10-05 21:30:08,307::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.HsmDomainMonitorLock', Clearing records. >Thread-20::INFO::2014-10-05 21:30:08,307::logUtils::47::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True >Thread-20::DEBUG::2014-10-05 21:30:08,307::task::1191::Storage.TaskManager.Task::(prepare) Task=`2b318ae5-5ba9-4d19-afb0-75ac53b14a13`::finished: True >Thread-20::DEBUG::2014-10-05 21:30:08,307::task::595::Storage.TaskManager.Task::(_updateState) Task=`2b318ae5-5ba9-4d19-afb0-75ac53b14a13`::moving from state preparing -> state finished >Thread-20::DEBUG::2014-10-05 21:30:08,308::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-20::DEBUG::2014-10-05 21:30:08,308::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-20::DEBUG::2014-10-05 21:30:08,308::task::993::Storage.TaskManager.Task::(_decref) Task=`2b318ae5-5ba9-4d19-afb0-75ac53b14a13`::ref 0 aborting False >Thread-20::DEBUG::2014-10-05 21:30:08,308::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.connect' in bridge with True >Thread-20::DEBUG::2014-10-05 21:30:08,308::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-21::DEBUG::2014-10-05 21:30:08,355::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-21::DEBUG::2014-10-05 21:30:08,355::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-21::DEBUG::2014-10-05 21:30:08,355::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-21::DEBUG::2014-10-05 21:30:08,355::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata count=1 (cwd None) >Thread-21::DEBUG::2014-10-05 21:30:08,360::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.00063402 s, 3.2 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-05 21:30:08,360::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.00063402 s, 3.2 MB/s'], size: 2048 >Thread-21::DEBUG::2014-10-05 21:30:08,360::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-21::DEBUG::2014-10-05 21:30:08,360::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-21::DEBUG::2014-10-05 21:30:08,360::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-21::DEBUG::2014-10-05 21:30:08,360::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-21::DEBUG::2014-10-05 21:30:08,361::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-21::DEBUG::2014-10-05 21:30:08,361::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-21::DEBUG::2014-10-05 21:30:08,361::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-21::DEBUG::2014-10-05 21:30:08,361::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-21::DEBUG::2014-10-05 21:30:08,361::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060001|/dev/mapper/1IET_00060002|/dev/mapper/1IET_00060003|/dev/mapper/1IET_00060004|/dev/mapper/1IET_00060005|/dev/mapper/1IET_00060006|/dev/mapper/1IET_00060007|/dev/mapper/1IET_00060008|/dev/mapper/1IET_00060009|/dev/mapper/1IET_0006000a|/dev/mapper/3600144f09dbd05000000522dde040011|/dev/mapper/3600144f09dbd05000000522dde050012|/dev/mapper/3600144f09dbd05000000522dde070013|/dev/mapper/3600144f09dbd05000000522dde090014|/dev/mapper/3600144f09dbd05000000522dde0a0015|/dev/mapper/3600144f09dbd05000000522dde0c0016|/dev/mapper/3600144f09dbd05000000522dde0d0017|/dev/mapper/3600144f09dbd05000000522dde0f0018|/dev/mapper/3600144f09dbd05000000522dde100019|/dev/mapper/3600144f09dbd05000000522dde12001a|/dev/mapper/3600144f09dbd0500000053a84cba0002|/dev/mapper/3600144f09dbd0500000053a84e730003|/dev/mapper/3600144f09dbd0500000053a84f4d0004|/dev/mapper/3600144f09dbd0500000053a8503b0005|/dev/mapper/3600144f09dbd0500000053a850500006|/dev/mapper/3600144f09dbd0500000053a8505b0007|/dev/mapper/3600144f09dbd0500000053a850640008|/dev/mapper/3600144f09dbd0500000053a8506c0009|/dev/mapper/3600144f09dbd0500000053a85076000a|/dev/mapper/3600144f09dbd0500000053a8507f000b|/dev/mapper/3600144f09dbd0500000053a853ed000c|/dev/mapper/3600144f09dbd0500000053a85401000d|/dev/mapper/3600144f09dbd0500000053a85409000e|/dev/mapper/3600144f09dbd0500000053a85410000f|/dev/mapper/3600144f09dbd0500000053a854180010|/dev/mapper/3600144f09dbd0500000053a854250011|/dev/mapper/3600144f09dbd0500000053a8542d0012|/dev/mapper/3600144f09dbd0500000053a8543e0013|/dev/mapper/3600144f09dbd0500000053a854450014|/dev/mapper/3600144f09dbd0500000053a8544d0015|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name c73a3ced-4b41-4b4e-93f7-494d63e7e7ae (cwd None) >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:30:08,393::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:30:08,395::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-31::DEBUG::2014-10-05 21:30:08,396::task::595::Storage.TaskManager.Task::(_updateState) Task=`f6a6a867-cd87-4df0-ad1e-002f6d7729e3`::moving from state init -> state preparing >Thread-31::INFO::2014-10-05 21:30:08,397::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-31::INFO::2014-10-05 21:30:08,397::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': -1, 'acquired': False, 'delay': '0', 'lastCheck': '0.4', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000451325', 'lastCheck': '0.2', 'valid': True}} >Thread-31::DEBUG::2014-10-05 21:30:08,397::task::1191::Storage.TaskManager.Task::(prepare) Task=`f6a6a867-cd87-4df0-ad1e-002f6d7729e3`::finished: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': -1, 'acquired': False, 'delay': '0', 'lastCheck': '0.4', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000451325', 'lastCheck': '0.2', 'valid': True}} >Thread-31::DEBUG::2014-10-05 21:30:08,397::task::595::Storage.TaskManager.Task::(_updateState) Task=`f6a6a867-cd87-4df0-ad1e-002f6d7729e3`::moving from state preparing -> state finished >Thread-31::DEBUG::2014-10-05 21:30:08,397::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-31::DEBUG::2014-10-05 21:30:08,397::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-31::DEBUG::2014-10-05 21:30:08,397::task::993::Storage.TaskManager.Task::(_decref) Task=`f6a6a867-cd87-4df0-ad1e-002f6d7729e3`::ref 0 aborting False >Thread-31::DEBUG::2014-10-05 21:30:08,399::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:30:08,489::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:30:08,490::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-32::DEBUG::2014-10-05 21:30:08,490::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.setMOMPolicyParameters' in bridge with {u'key_value_store': {u'balloonEnabled': True, u'ksmEnabled': True}} >Thread-32::DEBUG::2014-10-05 21:30:08,532::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.setMOMPolicyParameters' in bridge with True >Thread-32::DEBUG::2014-10-05 21:30:08,532::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-21::DEBUG::2014-10-05 21:30:08,569::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-21::DEBUG::2014-10-05 21:30:08,569::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-21::DEBUG::2014-10-05 21:30:08,569::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', u'PV1=pv:3600144f09dbd0500000053a84e730003,uuid:5zWyZx-cpAZ-tMON-3wa5-s6pN-Xj7N-b3fPs5,pestart:0,pecount:77,mapoffset:77', 'DESCRIPTION=sasha-el7', 'VERSION=3', u'PV2=pv:3600144f09dbd0500000053a853ed000c,uuid:c2LMVb-PXdj-XpTE-4f06-2rrh-TdAG-zzehcn,pestart:0,pecount:77,mapoffset:154', 'SDUUID=c73a3ced-4b41-4b4e-93f7-494d63e7e7ae', 'LOGBLKSIZE=512', 'TYPE=ISCSI', u'PV3=pv:3600144f09dbd0500000053a85401000d,uuid:1kMage-eAId-qy7G-oIqH-tfbH-ScLD-gqv7JM,pestart:0,pecount:77,mapoffset:231', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'PHYBLKSIZE=512', 'VGUUID=pF71zc-KPBX-BMKm-7KHK-KihY-ysp4-6DZTpA', 'CLASS=Data', 'LOCKPOLICY=', 'ROLE=Regular', u'PV0=pv:3600144f09dbd0500000053a85076000a,uuid:Ka3ewV-ddPQ-boL6-sDJT-xcFb-YBkc-9HeKFq,pestart:0,pecount:77,mapoffset:0', 'POOL_UUID=b86b687a-d073-497a-ac8a-249025419a3e', '_SHA_CKSUM=40993341560adfe664d3c1ab0c6b62068addc74c'] >Thread-21::DEBUG::2014-10-05 21:30:08,570::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/3600144f09dbd0500000053a84e730003|/dev/mapper/3600144f09dbd0500000053a85076000a|/dev/mapper/3600144f09dbd0500000053a853ed000c|/dev/mapper/3600144f09dbd0500000053a85401000d|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --available y c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/ids (cwd None) >Thread-21::DEBUG::2014-10-05 21:30:08,711::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-21::DEBUG::2014-10-05 21:30:08,711::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-21::DEBUG::2014-10-05 21:30:08,711::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-21::DEBUG::2014-10-05 21:30:08,711::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/3600144f09dbd0500000053a84e730003|/dev/mapper/3600144f09dbd0500000053a85076000a|/dev/mapper/3600144f09dbd0500000053a853ed000c|/dev/mapper/3600144f09dbd0500000053a85401000d|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --available y c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/leases (cwd None) >Thread-21::DEBUG::2014-10-05 21:30:08,885::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-21::DEBUG::2014-10-05 21:30:08,885::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-21::DEBUG::2014-10-05 21:30:08,886::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-21::DEBUG::2014-10-05 21:30:08,886::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/3600144f09dbd0500000053a84e730003|/dev/mapper/3600144f09dbd0500000053a85076000a|/dev/mapper/3600144f09dbd0500000053a853ed000c|/dev/mapper/3600144f09dbd0500000053a85401000d|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --available y c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/inbox c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/outbox c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/master (cwd None) >Thread-21::DEBUG::2014-10-05 21:30:09,022::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-21::DEBUG::2014-10-05 21:30:09,023::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-21::DEBUG::2014-10-05 21:30:09,023::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-21::DEBUG::2014-10-05 21:30:09,023::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae_imageNS' >Thread-21::DEBUG::2014-10-05 21:30:09,024::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae_volumeNS' >Thread-21::DEBUG::2014-10-05 21:30:09,024::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae_lvmActivationNS' >Thread-21::DEBUG::2014-10-05 21:30:09,024::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgck --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/3600144f09dbd0500000053a84e730003|/dev/mapper/3600144f09dbd0500000053a85076000a|/dev/mapper/3600144f09dbd0500000053a853ed000c|/dev/mapper/3600144f09dbd0500000053a85401000d|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' c73a3ced-4b41-4b4e-93f7-494d63e7e7ae (cwd None) >Thread-21::DEBUG::2014-10-05 21:30:09,120::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-21::DEBUG::2014-10-05 21:30:09,121::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-10-05 21:30:09,125::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000528577 s, 7.7 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-05 21:30:09,127::domainMonitor::264::Storage.DomainMonitorThread::(_monitorDomain) Domain c73a3ced-4b41-4b4e-93f7-494d63e7e7ae changed its status to Valid >Thread-33::DEBUG::2014-10-05 21:30:09,127::misc::821::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Emitting event >Thread-21::INFO::2014-10-05 21:30:09,127::clusterlock::222::Storage.SANLock::(acquireHostId) Acquiring host id for domain c73a3ced-4b41-4b4e-93f7-494d63e7e7ae (id: 3) >Thread-33::DEBUG::2014-10-05 21:30:09,128::misc::831::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `contEIOVms` >Thread-21::DEBUG::2014-10-05 21:30:09,129::clusterlock::240::Storage.SANLock::(acquireHostId) Host id for domain c73a3ced-4b41-4b4e-93f7-494d63e7e7ae successfully acquired (id: 3) >Thread-33::DEBUG::2014-10-05 21:30:09,129::misc::841::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Event emitted >Thread-34::INFO::2014-10-05 21:30:09,130::clientIF::136::vds::(contEIOVms) vmContainerLock acquired >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:30:11,686::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:30:11,687::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-35::DEBUG::2014-10-05 21:30:11,688::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:30:14,778::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:30:14,779::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-36::DEBUG::2014-10-05 21:30:14,781::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:30:17,871::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:30:17,871::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-37::DEBUG::2014-10-05 21:30:17,872::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-22::DEBUG::2014-10-05 21:30:18,155::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-10-05 21:30:18,160::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000725115 s, 5.6 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-05 21:30:19,130::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-10-05 21:30:19,139::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000587805 s, 7.0 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:30:20,963::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:30:20,964::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-38::DEBUG::2014-10-05 21:30:20,966::task::595::Storage.TaskManager.Task::(_updateState) Task=`de2a54e2-9464-44e8-872e-11b466fea4a3`::moving from state init -> state preparing >Thread-38::INFO::2014-10-05 21:30:20,966::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-38::INFO::2014-10-05 21:30:20,966::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000587805', 'lastCheck': '1.8', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000725115', 'lastCheck': '2.8', 'valid': True}} >Thread-38::DEBUG::2014-10-05 21:30:20,966::task::1191::Storage.TaskManager.Task::(prepare) Task=`de2a54e2-9464-44e8-872e-11b466fea4a3`::finished: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000587805', 'lastCheck': '1.8', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000725115', 'lastCheck': '2.8', 'valid': True}} >Thread-38::DEBUG::2014-10-05 21:30:20,966::task::595::Storage.TaskManager.Task::(_updateState) Task=`de2a54e2-9464-44e8-872e-11b466fea4a3`::moving from state preparing -> state finished >Thread-38::DEBUG::2014-10-05 21:30:20,966::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-38::DEBUG::2014-10-05 21:30:20,966::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-38::DEBUG::2014-10-05 21:30:20,966::task::993::Storage.TaskManager.Task::(_decref) Task=`de2a54e2-9464-44e8-872e-11b466fea4a3`::ref 0 aborting False >Thread-38::DEBUG::2014-10-05 21:30:20,968::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:30:21,062::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:30:21,063::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-39::DEBUG::2014-10-05 21:30:21,064::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:30:24,195::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:30:24,197::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-40::DEBUG::2014-10-05 21:30:24,198::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:30:27,285::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:30:27,285::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-41::DEBUG::2014-10-05 21:30:27,286::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-22::DEBUG::2014-10-05 21:30:28,162::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-22::DEBUG::2014-10-05 21:30:28,167::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000741525 s, 5.5 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-05 21:30:29,141::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-21::DEBUG::2014-10-05 21:30:29,145::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000571981 s, 7.2 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:30:30,374::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:30:30,375::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-42::DEBUG::2014-10-05 21:30:30,376::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:30:33,465::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:30:33,465::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-43::DEBUG::2014-10-05 21:30:33,466::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >MainThread::DEBUG::2014-10-05 21:30:34,135::vdsm::58::vds::(sigtermHandler) Received signal 15 >MainThread::DEBUG::2014-10-05 21:30:34,143::protocoldetector::135::vds.MultiProtocolAcceptor::(stop) Stopping Acceptor >MainThread::INFO::2014-10-05 21:30:34,144::__init__::563::jsonrpc.JsonRpcServer::(stop) Stopping JsonRPC Server >Detector thread::DEBUG::2014-10-05 21:30:34,144::protocoldetector::106::vds.MultiProtocolAcceptor::(_cleanup) Cleaning Acceptor >MainThread::INFO::2014-10-05 21:30:34,147::vmchannels::188::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2014-10-05 21:30:34,147::momIF::91::MOM::(stop) Shutting down MOM >MainThread::DEBUG::2014-10-05 21:30:34,148::task::595::Storage.TaskManager.Task::(_updateState) Task=`36ad8c61-1419-4478-ade5-98cd493887b9`::moving from state init -> state preparing >MainThread::INFO::2014-10-05 21:30:34,148::logUtils::44::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2014-10-05 21:30:34,149::storageServer::706::Storage.ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2014-10-05 21:30:34,152::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/04511d19-f03a-469a-9e31-ab51d6e75a56/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:30:34,152::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/67ba7199-44ec-4562-9c17-ebc3903c7c7f/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:30:34,153::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/83d26f74-2a29-4025-8210-5ab61cb49a82/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:30:34,153::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/fd98c9a9-23f9-43f5-8e7b-4022f5bdd683/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:30:34,154::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/b9c2b835-cdbd-48ea-8d69-7a33042c79b1/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:30:34,154::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/c175b488-c93a-4dff-b709-b137853eab67/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:30:34,154::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/6ca7f5aa-6229-4549-9c6f-54ec7b7099a6/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:30:34,155::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/681bd5bc-4d98-4da3-bd9a-cd37eec7b9af/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:30:34,155::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050/master` is not mounted, skipping >Thread-30::INFO::2014-10-05 21:30:34,308::storage_mailbox::509::Storage.MailBox.HsmMailMonitor::(run) HSM_MailboxMonitor - Incoming mail monitoring thread stopped, clearing outgoing mail >Thread-30::INFO::2014-10-05 21:30:34,308::storage_mailbox::385::Storage.MailBox.HsmMailMonitor::(_sendMail) HSM_MailMonitor sending mail to SPM - ['/bin/dd', 'of=/rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/mastersd/dom_md/inbox', 'iflag=fullblock', 'oflag=direct', 'conv=notrunc', 'bs=512', 'seek=24'] >Thread-30::DEBUG::2014-10-05 21:30:34,308::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /bin/dd of=/rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/mastersd/dom_md/inbox iflag=fullblock oflag=direct conv=notrunc bs=512 seek=24 (cwd None) >Thread-30::DEBUG::2014-10-05 21:30:34,317::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.00377739 s, 1.1 MB/s\n'; <rc> = 0 >VM Channels Listener::INFO::2014-10-05 21:30:35,076::vmchannels::183::vds::(run) VM channels listener thread has ended. >MainThread::INFO::2014-10-05 21:30:35,291::domainMonitor::137::Storage.DomainMonitor::(close) Stopping domain monitors >MainThread::INFO::2014-10-05 21:30:35,291::domainMonitor::114::Storage.DomainMonitor::(stopMonitoring) Stop monitoring c73a3ced-4b41-4b4e-93f7-494d63e7e7ae >Thread-21::DEBUG::2014-10-05 21:30:35,291::domainMonitor::192::Storage.DomainMonitorThread::(_monitorLoop) Stopping domain monitor for c73a3ced-4b41-4b4e-93f7-494d63e7e7ae >Thread-21::INFO::2014-10-05 21:30:35,291::clusterlock::245::Storage.SANLock::(releaseHostId) Releasing host id for domain c73a3ced-4b41-4b4e-93f7-494d63e7e7ae (id: 3) >Thread-21::DEBUG::2014-10-05 21:30:37,297::clusterlock::255::Storage.SANLock::(releaseHostId) Host id for domain c73a3ced-4b41-4b4e-93f7-494d63e7e7ae released successfully (id: 3) >MainThread::INFO::2014-10-05 21:30:37,298::domainMonitor::114::Storage.DomainMonitor::(stopMonitoring) Stop monitoring ff559f46-c495-4f6b-901c-2a624042a050 >Thread-22::DEBUG::2014-10-05 21:30:37,298::domainMonitor::192::Storage.DomainMonitorThread::(_monitorLoop) Stopping domain monitor for ff559f46-c495-4f6b-901c-2a624042a050 >Thread-22::INFO::2014-10-05 21:30:37,298::clusterlock::245::Storage.SANLock::(releaseHostId) Releasing host id for domain ff559f46-c495-4f6b-901c-2a624042a050 (id: 3) >Thread-22::DEBUG::2014-10-05 21:30:39,299::clusterlock::255::Storage.SANLock::(releaseHostId) Host id for domain ff559f46-c495-4f6b-901c-2a624042a050 released successfully (id: 3) >MainThread::DEBUG::2014-10-05 21:30:39,299::taskManager::90::Storage.TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2014-10-05 21:30:39,299::logUtils::47::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2014-10-05 21:30:39,299::task::1191::Storage.TaskManager.Task::(prepare) Task=`36ad8c61-1419-4478-ade5-98cd493887b9`::finished: None >MainThread::DEBUG::2014-10-05 21:30:39,299::task::595::Storage.TaskManager.Task::(_updateState) Task=`36ad8c61-1419-4478-ade5-98cd493887b9`::moving from state preparing -> state finished >MainThread::DEBUG::2014-10-05 21:30:39,300::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2014-10-05 21:30:39,300::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2014-10-05 21:30:39,300::task::993::Storage.TaskManager.Task::(_decref) Task=`36ad8c61-1419-4478-ade5-98cd493887b9`::ref 0 aborting False >MainThread::INFO::2014-10-05 21:30:39,300::vdsm::137::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2014-10-05 21:30:39,300::vdsm::141::vds::(run) <_MainThread(MainThread, started 140119674844928)> >MainThread::INFO::2014-10-05 21:30:39,300::vdsm::141::vds::(run) <Thread(libvirtEventLoop, started daemon 140119586518784)> >MainThread::INFO::2014-10-05 21:30:39,300::vdsm::141::vds::(run) <WorkerThread(Thread-2, started daemon 140119565539072)> >MainThread::INFO::2014-10-05 21:30:39,300::vdsm::141::vds::(run) <PolicyEngine(PolicyEngine, started daemon 140117805102848)> >MainThread::INFO::2014-10-05 21:30:39,300::vdsm::141::vds::(run) <WorkerThread(Thread-8, started daemon 140119296157440)> >MainThread::INFO::2014-10-05 21:30:39,300::vdsm::141::vds::(run) <WorkerThread(Thread-7, started daemon 140119306647296)> >MainThread::INFO::2014-10-05 21:30:39,300::vdsm::141::vds::(run) <WorkerThread(Thread-4, started daemon 140119338116864)> >MainThread::INFO::2014-10-05 21:30:39,300::vdsm::141::vds::(run) <WorkerThread(Thread-6, started daemon 140119317137152)> >MainThread::INFO::2014-10-05 21:30:39,300::vdsm::141::vds::(run) <WorkerThread(Thread-5, started daemon 140119327627008)> >MainThread::INFO::2014-10-05 21:30:39,300::vdsm::141::vds::(run) <WorkerThread(Thread-10, started daemon 140118935463680)> >MainThread::INFO::2014-10-05 21:30:39,300::vdsm::141::vds::(run) <WorkerThread(Thread-3, started daemon 140119348606720)> >MainThread::INFO::2014-10-05 21:30:39,300::vdsm::141::vds::(run) <WorkerThread(Thread-9, started daemon 140118945953536)> >MainThread::INFO::2014-10-05 21:30:39,301::momIF::91::MOM::(stop) Shutting down MOM >MainThread::INFO::2014-10-05 21:30:39,301::vdsm::141::vds::(run) <MomThread(MOM, started 140118586042112)> >MainThread::INFO::2014-10-05 21:30:39,301::vdsm::141::vds::(run) <WorkerThread(Thread-1, started daemon 140119576028928)> >MainThread::INFO::2014-10-05 21:31:25,099::vdsm::131::vds::(run) (PID: 3057) I am the actual vdsm 4.16.6-0.el6 voodoo1.tlv.redhat.com (2.6.32-431.29.2.el6.x86_64) >MainThread::DEBUG::2014-10-05 21:31:25,118::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2014-10-05 21:31:25,118::threadPool::35::Storage.ThreadPool::(__init__) Enter - numThreads: 10, waitTimeout: 3, maxTasks: 500 >MainThread::DEBUG::2014-10-05 21:31:25,120::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt >MainThread::WARNING::2014-10-05 21:31:25,121::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt already exists >MainThread::DEBUG::2014-10-05 21:31:25,127::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/04511d19-f03a-469a-9e31-ab51d6e75a56/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:31:25,127::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/67ba7199-44ec-4562-9c17-ebc3903c7c7f/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:31:25,128::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/83d26f74-2a29-4025-8210-5ab61cb49a82/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:31:25,128::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/fd98c9a9-23f9-43f5-8e7b-4022f5bdd683/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:31:25,129::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/b9c2b835-cdbd-48ea-8d69-7a33042c79b1/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:31:25,129::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/c175b488-c93a-4dff-b709-b137853eab67/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:31:25,129::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/6ca7f5aa-6229-4549-9c6f-54ec7b7099a6/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:31:25,130::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/681bd5bc-4d98-4da3-bd9a-cd37eec7b9af/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:31:25,130::sp::381::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050/master` is not mounted, skipping >MainThread::DEBUG::2014-10-05 21:31:25,181::supervdsm::77::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2014-10-05 21:31:25,193::multipath::142::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2014-10-05 21:31:25,194::hsm::418::Storage.Misc.excCmd::(__validateLvmLockingType) /usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type (cwd None) >MainThread::DEBUG::2014-10-05 21:31:25,293::hsm::418::Storage.Misc.excCmd::(__validateLvmLockingType) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2014-10-05 21:31:25,293::hsm::442::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >MainThread::DEBUG::2014-10-05 21:31:25,294::hsm::474::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >MainThread::DEBUG::2014-10-05 21:31:25,294::hsm::475::Storage.HSM::(__cleanStorageRepository) Mount list: [] >MainThread::DEBUG::2014-10-05 21:31:25,294::hsm::477::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >MainThread::DEBUG::2014-10-05 21:31:25,297::hsm::520::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >storageRefresh::DEBUG::2014-10-05 21:31:25,297::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2014-10-05 21:31:25,299::dispatcher::46::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >storageRefresh::DEBUG::2014-10-05 21:31:25,299::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >storageRefresh::DEBUG::2014-10-05 21:31:25,303::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >storageRefresh::DEBUG::2014-10-05 21:31:25,317::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >storageRefresh::DEBUG::2014-10-05 21:31:25,317::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >storageRefresh::DEBUG::2014-10-05 21:31:25,317::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >storageRefresh::DEBUG::2014-10-05 21:31:25,327::misc::751::Storage.SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2014-10-05 21:31:25,361::task::595::Storage.TaskManager.Task::(_updateState) Task=`f4a46ca6-742c-4173-b02b-36f06aa117e4`::moving from state init -> state preparing >MainThread::INFO::2014-10-05 21:31:25,362::logUtils::44::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback(callbackFunc=<bound method clientIF.contEIOVms of <clientIF.clientIF object at 0x2ac04d0>>) >MainThread::INFO::2014-10-05 21:31:25,362::logUtils::47::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback, Return response: None >MainThread::DEBUG::2014-10-05 21:31:25,362::task::1191::Storage.TaskManager.Task::(prepare) Task=`f4a46ca6-742c-4173-b02b-36f06aa117e4`::finished: None >MainThread::DEBUG::2014-10-05 21:31:25,362::task::595::Storage.TaskManager.Task::(_updateState) Task=`f4a46ca6-742c-4173-b02b-36f06aa117e4`::moving from state preparing -> state finished >MainThread::DEBUG::2014-10-05 21:31:25,362::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2014-10-05 21:31:25,362::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2014-10-05 21:31:25,362::task::993::Storage.TaskManager.Task::(_decref) Task=`f4a46ca6-742c-4173-b02b-36f06aa117e4`::ref 0 aborting False >storageRefresh::DEBUG::2014-10-05 21:31:25,363::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >MainThread::INFO::2014-10-05 21:31:25,368::momIF::49::MOM::(__init__) Starting up MOM >MainThread::INFO::2014-10-05 21:31:25,373::vmchannels::192::vds::(settimeout) Setting channels' timeout to 30 seconds. >storageRefresh::DEBUG::2014-10-05 21:31:25,373::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >VM Channels Listener::INFO::2014-10-05 21:31:25,377::vmchannels::175::vds::(run) Starting VM channels listener thread. >clientIFinit::DEBUG::2014-10-05 21:31:25,377::libvirtconnection::150::root::(get) trying to connect libvirt >storageRefresh::DEBUG::2014-10-05 21:31:25,378::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >storageRefresh::DEBUG::2014-10-05 21:31:25,378::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-10-05 21:31:25,378::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >storageRefresh::DEBUG::2014-10-05 21:31:25,383::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-10-05 21:31:25,385::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >MainThread::DEBUG::2014-10-05 21:31:25,388::protocoldetector::131::vds.MultiProtocolAcceptor::(add_detector) adding detector: <rpc.BindingXMLRPC.XmlDetector instance at 0x2bb7830> >storageRefresh::DEBUG::2014-10-05 21:31:25,388::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-10-05 21:31:25,395::misc::751::Storage.SamplingMethod::(__call__) Returning last result >storageRefresh::DEBUG::2014-10-05 21:31:25,396::lvm::317::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >storageRefresh::DEBUG::2014-10-05 21:31:25,422::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm pvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size (cwd None) >MainThread::DEBUG::2014-10-05 21:31:25,447::protocoldetector::131::vds.MultiProtocolAcceptor::(add_detector) adding detector: <yajsonrpc.stompReactor.StompDetector instance at 0x2d44560> >JsonRpcServer::DEBUG::2014-10-05 21:31:25,448::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Detector thread::DEBUG::2014-10-05 21:31:25,449::protocoldetector::70::vds.MultiProtocolAcceptor::(serve_forever) Acceptor running >Detector thread::DEBUG::2014-10-05 21:31:25,450::protocoldetector::72::vds.MultiProtocolAcceptor::(serve_forever) Using required_size=7 >storageRefresh::DEBUG::2014-10-05 21:31:25,503::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >storageRefresh::DEBUG::2014-10-05 21:31:25,504::lvm::342::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >storageRefresh::DEBUG::2014-10-05 21:31:25,505::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >storageRefresh::DEBUG::2014-10-05 21:31:25,505::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >storageRefresh::DEBUG::2014-10-05 21:31:25,586::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-10-05 21:31:25,586::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >storageRefresh::DEBUG::2014-10-05 21:31:25,586::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags (cwd None) >storageRefresh::DEBUG::2014-10-05 21:31:25,659::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-10-05 21:31:25,659::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >storageRefresh::DEBUG::2014-10-05 21:31:25,660::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >storageRefresh::DEBUG::2014-10-05 21:31:25,734::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-10-05 21:31:25,734::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >storageRefresh::DEBUG::2014-10-05 21:31:25,734::hsm::387::Storage.HSM::(storageRefresh) HSM is ready >Detector thread::DEBUG::2014-10-05 21:33:40,437::protocoldetector::166::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.201.227:58604 >Detector thread::DEBUG::2014-10-05 21:33:40,521::protocoldetector::177::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.201.227:58604 >Detector thread::DEBUG::2014-10-05 21:33:40,521::protocoldetector::203::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol stomp from 10.35.201.227:58604 >Detector thread::DEBUG::2014-10-05 21:33:40,521::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='CONNECT'> >Detector thread::INFO::2014-10-05 21:33:40,521::stompReactor::66::Broker.StompAdapter::(_cmd_connect) Processing CONNECT request >Detector thread::DEBUG::2014-10-05 21:33:40,522::stompReactor::338::protocoldetector.StompDetector::(handleSocket) Stomp detected from ('10.35.201.227', 58604) >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:33:40,603::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SUBSCRIBE'> >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:33:40,603::stompReactor::89::Broker.StompAdapter::(_cmd_subscribe) Subscribe command ignored >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:33:40,727::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-13::DEBUG::2014-10-05 21:33:40,728::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getCapabilities' in bridge with {} >JsonRpcServer::DEBUG::2014-10-05 21:33:40,728::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-13::DEBUG::2014-10-05 21:33:40,769::utils::738::root::(execCmd) /sbin/ip route show to 0.0.0.0/0 table all (cwd None) >Thread-13::DEBUG::2014-10-05 21:33:40,775::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:33:40,786::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/bonding_masters (cwd None) >Thread-13::DEBUG::2014-10-05 21:33:40,805::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:33:40,805::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/Qv0acGggSVQF1ev/bonding/mode (cwd None) >Thread-13::DEBUG::2014-10-05 21:33:40,812::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:33:40,813::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/Qv0acGggSVQF1ev/bonding/mode (cwd None) >Thread-13::DEBUG::2014-10-05 21:33:40,820::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:33:40,821::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/Qv0acGggSVQF1ev/bonding/mode (cwd None) >Thread-13::DEBUG::2014-10-05 21:33:40,834::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:33:40,835::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/Qv0acGggSVQF1ev/bonding/mode (cwd None) >Thread-13::DEBUG::2014-10-05 21:33:40,842::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:33:40,843::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/Qv0acGggSVQF1ev/bonding/mode (cwd None) >Thread-13::DEBUG::2014-10-05 21:33:40,850::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:33:40,851::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/Qv0acGggSVQF1ev/bonding/mode (cwd None) >Thread-13::DEBUG::2014-10-05 21:33:40,860::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:33:40,861::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/Qv0acGggSVQF1ev/bonding/mode (cwd None) >Thread-13::DEBUG::2014-10-05 21:33:40,868::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:33:40,869::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/bonding_masters (cwd None) >Thread-13::DEBUG::2014-10-05 21:33:40,907::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:33:40,962::caps::646::root::(get) VirtioRNG DISABLED: libvirt version 0.10.2-29.el6_5.12 required >= 0.10.2-31 >Thread-13::DEBUG::2014-10-05 21:33:40,963::utils::738::root::(execCmd) numactl --hardware (cwd None) >Thread-13::DEBUG::2014-10-05 21:33:40,966::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:33:40,967::utils::738::root::(execCmd) sysctl -n -e kernel.numa_balancing (cwd None) >Thread-13::DEBUG::2014-10-05 21:33:40,969::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-13::DEBUG::2014-10-05 21:33:40,971::caps::269::root::(getLiveMergeSupport) libvirt is missing 'VIR_DOMAIN_BLOCK_COMMIT_RELATIVE': live merge disabled >Thread-13::DEBUG::2014-10-05 21:33:40,973::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.getCapabilities' in bridge with {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:nsoffer1'}], 'FC': []}, 'packages2': {'kernel': {'release': '431.29.2.el6.x86_64', 'buildtime': 1406465746.0, 'version': '2.6.32'}, 'spice-server': {'release': '6.el6_5.2', 'buildtime': 1401730350L, 'version': '0.12.4'}, 'vdsm': {'release': '0.el6', 'buildtime': 1412232469L, 'version': '4.16.6'}, 'qemu-kvm': {'release': '2.415.el6_5.14', 'buildtime': 1408575896L, 'version': '0.12.1.2'}, 'libvirt': {'release': '29.el6_5.12', 'buildtime': 1408610144L, 'version': '0.10.2'}, 'qemu-img': {'release': '2.415.el6_5.14', 'buildtime': 1408575896L, 'version': '0.12.1.2'}, 'mom': {'release': '0.0.master.20140911.gitcf7ff17.el6', 'buildtime': 1410462635L, 'version': '0.4.2'}}, 'numaNodeDistance': {'0': [10]}, 'cpuModel': 'Intel Core Processor (Haswell)', 'liveMerge': 'false', 'hooks': {}, 'cpuSockets': '8', 'vmTypes': ['kvm'], 'selinux': {'mode': '1'}, 'kdumpStatus': 1, 'supportedProtocols': ['2.2', '2.3'], 'networks': {'ovirtmgmt': {'iface': 'ovirtmgmt', 'addr': '10.35.0.100', 'cfg': {'DEFROUTE': 'yes', 'HOTPLUG': 'no', 'MTU': '1500', 'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'STP': 'off', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'bridged': True, 'ipv6addrs': ['fe80::5054:ff:fe87:196f/64'], 'gateway': '10.35.1.254', 'bootproto4': 'dhcp', 'netmask': '255.255.254.0', 'stp': 'off', 'ipv4addrs': ['10.35.0.100/23'], 'mtu': '1500', 'ipv6gateway': '::', 'ports': ['eth0']}}, 'bridges': {'ovirtmgmt': {'addr': '10.35.0.100', 'cfg': {'DEFROUTE': 'yes', 'HOTPLUG': 'no', 'MTU': '1500', 'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'STP': 'off', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'ipv6addrs': ['fe80::5054:ff:fe87:196f/64'], 'mtu': '1500', 'netmask': '255.255.254.0', 'stp': 'off', 'ipv4addrs': ['10.35.0.100/23'], 'ipv6gateway': '::', 'gateway': '10.35.1.254', 'opts': {'topology_change_detected': '0', 'multicast_last_member_count': '2', 'hash_elasticity': '4', 'multicast_query_response_interval': '999', 'multicast_snooping': '1', 'multicast_startup_query_interval': '3124', 'hello_timer': '161', 'multicast_querier_interval': '25496', 'max_age': '1999', 'hash_max': '512', 'stp_state': '0', 'root_id': '8000.52540087196f', 'priority': '32768', 'multicast_membership_interval': '25996', 'root_path_cost': '0', 'root_port': '0', 'multicast_querier': '0', 'multicast_startup_query_count': '2', 'hello_time': '199', 'topology_change': '0', 'bridge_id': '8000.52540087196f', 'topology_change_timer': '0', 'ageing_time': '29995', 'gc_timer': '61', 'group_addr': '1:80:c2:0:0:0', 'tcn_timer': '0', 'multicast_query_interval': '12498', 'multicast_last_member_interval': '99', 'multicast_router': '1', 'forward_delay': '0'}, 'ports': ['eth0']}}, 'uuid': '85CFDED0-FD24-4645-AF7C-D47A088AE9E8', 'lastClientIface': 'ovirtmgmt', 'nics': {'eth0': {'addr': '', 'cfg': {'BRIDGE': 'ovirtmgmt', 'NM_CONTROLLED': 'no', 'MTU': '1500', 'HWADDR': '52:54:00:87:19:6f', 'DEVICE': 'eth0', 'ONBOOT': 'yes'}, 'ipv6addrs': [], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': '52:54:00:87:19:6f', 'speed': 0}}, 'software_revision': '0', 'clusterLevels': ['3.0', '3.1', '3.2', '3.3', '3.4', '3.5'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,mmx,fxsr,sse,sse2,ss,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,rep_good,unfair_spinlock,pni,pclmulqdq,vmx,ssse3,fma,cx16,pcid,sse4_1,sse4_2,x2apic,movbe,popcnt,tsc_deadline_timer,aes,xsave,avx,f16c,rdrand,hypervisor,lahf_lm,abm,xsaveopt,vnmi,ept,fsgsbase,bmi1,hle,avx2,smep,bmi2,erms,invpcid,rtm,model_Haswell,model_Nehalem,model_Conroe,model_Penryn,model_Westmere,model_SandyBridge', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:nsoffer1', 'netConfigDirty': 'False', 'supportedENGINEs': ['3.0', '3.1', '3.2', '3.3', '3.4', '3.5'], 'autoNumaBalancing': 2, 'reservedMem': '321', 'bondings': {'bond4': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond0': {'addr': '', 'cfg': {}, 'ipv6addrs': [], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': '00:00:00:00:00:00', 'slaves': [], 'opts': {}}, 'bond1': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond2': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond3': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}}, 'software_version': '4.16', 'memSize': '3734', 'cpuSpeed': '3392.160', 'numaNodes': {u'0': {'totalMemory': '3734', 'cpus': [0, 1, 2, 3, 4, 5, 6, 7]}}, 'version_name': 'Snow Man', 'vlans': {}, 'cpuCores': '8', 'kvmEnabled': 'true', 'guestOverhead': '65', 'cpuThreads': '8', 'emulatedMachines': [u'rhel6.5.0', u'pc', u'rhel6.4.0', u'rhel6.3.0', u'rhel6.2.0', u'rhel6.1.0', u'rhel6.0.0', u'rhel5.5.0', u'rhel5.4.4', u'rhel5.4.0'], 'operatingSystem': {'release': '6.5.0.1.el6', 'version': '6Server', 'name': 'RHEL'}} >Thread-13::DEBUG::2014-10-05 21:33:40,975::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:33:41,071::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:33:41,072::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-14::DEBUG::2014-10-05 21:33:41,072::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getHardwareInfo' in bridge with {} >Thread-14::DEBUG::2014-10-05 21:33:41,119::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.getHardwareInfo' in bridge with {'systemProductName': 'Bochs', 'systemUUID': '85cfded0-fd24-4645-af7c-d47a088ae9e8', 'systemManufacturer': 'Bochs'} >Thread-14::DEBUG::2014-10-05 21:33:41,119::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:33:41,235::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:33:41,236::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-15::DEBUG::2014-10-05 21:33:41,237::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:33:41,355::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:33:41,356::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-16::DEBUG::2014-10-05 21:33:41,356::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.connectStorageServer' in bridge with {u'connectionParams': [{u'id': u'af150e8e-4d74-4810-b8bf-0c6a19637dbf', u'connection': u'10.35.16.25', u'iqn': u'iqn.1994-05.com.redhat:nsoffer-target1', u'user': u'', u'tpgt': u'1', u'password': u'', u'port': u'3260'}, {u'id': u'e8d58651-6414-4fc2-b281-2141cc97829b', u'connection': u'10.35.0.98', u'iqn': u'iqn.1994-05.com.redhat:target6', u'user': u'', u'tpgt': u'1', u'password': u'', u'port': u'3260'}, {u'id': u'8f399b5f-1237-479e-82e6-eedb202b66fd', u'connection': u'10.35.16.25', u'iqn': u'iqn.1994-05.com.redhat:nsoffer-target2', u'user': u'', u'tpgt': u'1', u'password': u'', u'port': u'3260'}], u'storagepoolID': u'b86b687a-d073-497a-ac8a-249025419a3e', u'domainType': 3} >Thread-16::DEBUG::2014-10-05 21:33:41,357::task::595::Storage.TaskManager.Task::(_updateState) Task=`cb501390-cb62-4322-be31-6247184b7542`::moving from state init -> state preparing >Thread-16::INFO::2014-10-05 21:33:41,357::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=3, spUUID=u'b86b687a-d073-497a-ac8a-249025419a3e', conList=[{u'connection': u'10.35.16.25', u'iqn': u'iqn.1994-05.com.redhat:nsoffer-target1', u'user': u'', u'tpgt': u'1', u'password': '******', u'id': u'af150e8e-4d74-4810-b8bf-0c6a19637dbf', u'port': u'3260'}, {u'connection': u'10.35.0.98', u'iqn': u'iqn.1994-05.com.redhat:target6', u'user': u'', u'tpgt': u'1', u'password': '******', u'id': u'e8d58651-6414-4fc2-b281-2141cc97829b', u'port': u'3260'}, {u'connection': u'10.35.16.25', u'iqn': u'iqn.1994-05.com.redhat:nsoffer-target2', u'user': u'', u'tpgt': u'1', u'password': '******', u'id': u'8f399b5f-1237-479e-82e6-eedb202b66fd', u'port': u'3260'}], options=None) >Thread-16::DEBUG::2014-10-05 21:33:41,358::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:nsoffer-target1 -I default -p 10.35.16.25:3260,1 --op=new (cwd None) >Thread-16::DEBUG::2014-10-05 21:33:41,370::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-10-05 21:33:41,370::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:nsoffer-target1 -I default -p 10.35.16.25:3260,1 -l (cwd None) >Thread-16::DEBUG::2014-10-05 21:33:42,167::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-10-05 21:33:42,168::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:nsoffer-target1 -I default -p 10.35.16.25:3260,1 -n node.startup -v manual --op=update (cwd None) >Thread-16::DEBUG::2014-10-05 21:33:42,177::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-10-05 21:33:42,177::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-16::DEBUG::2014-10-05 21:33:42,178::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-16::DEBUG::2014-10-05 21:33:42,326::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-05 21:33:42,328::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-16::DEBUG::2014-10-05 21:33:42,328::hsm::2396::Storage.HSM::(__prefetchDomains) Found SD uuids: () >Thread-16::DEBUG::2014-10-05 21:33:42,328::hsm::2452::Storage.HSM::(connectStorageServer) knownSDs: {} >Thread-16::DEBUG::2014-10-05 21:33:42,328::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:target6 -I default -p 10.35.0.98:3260,1 --op=new (cwd None) >Thread-16::DEBUG::2014-10-05 21:33:42,378::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-10-05 21:33:42,378::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:target6 -I default -p 10.35.0.98:3260,1 -l (cwd None) >Thread-16::DEBUG::2014-10-05 21:33:42,968::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-10-05 21:33:42,968::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:target6 -I default -p 10.35.0.98:3260,1 -n node.startup -v manual --op=update (cwd None) >Thread-16::DEBUG::2014-10-05 21:33:42,983::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-10-05 21:33:42,983::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-16::DEBUG::2014-10-05 21:33:42,984::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-16::DEBUG::2014-10-05 21:33:43,114::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-05 21:33:43,114::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-16::DEBUG::2014-10-05 21:33:43,115::hsm::2396::Storage.HSM::(__prefetchDomains) Found SD uuids: () >Thread-16::DEBUG::2014-10-05 21:33:43,115::hsm::2452::Storage.HSM::(connectStorageServer) knownSDs: {} >Thread-16::DEBUG::2014-10-05 21:33:43,116::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:nsoffer-target2 -I default -p 10.35.16.25:3260,1 --op=new (cwd None) >Thread-16::DEBUG::2014-10-05 21:33:43,229::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-10-05 21:33:43,229::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:nsoffer-target2 -I default -p 10.35.16.25:3260,1 -l (cwd None) >Thread-16::DEBUG::2014-10-05 21:33:44,027::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-10-05 21:33:44,028::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1994-05.com.redhat:nsoffer-target2 -I default -p 10.35.16.25:3260,1 -n node.startup -v manual --op=update (cwd None) >Thread-16::DEBUG::2014-10-05 21:33:44,037::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-10-05 21:33:44,038::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-16::DEBUG::2014-10-05 21:33:44,038::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-16::DEBUG::2014-10-05 21:33:44,137::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-05 21:33:44,137::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-16::DEBUG::2014-10-05 21:33:44,137::hsm::2396::Storage.HSM::(__prefetchDomains) Found SD uuids: () >Thread-16::DEBUG::2014-10-05 21:33:44,138::hsm::2452::Storage.HSM::(connectStorageServer) knownSDs: {} >Thread-16::INFO::2014-10-05 21:33:44,139::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': u'af150e8e-4d74-4810-b8bf-0c6a19637dbf'}, {'status': 0, 'id': u'e8d58651-6414-4fc2-b281-2141cc97829b'}, {'status': 0, 'id': u'8f399b5f-1237-479e-82e6-eedb202b66fd'}]} >Thread-16::DEBUG::2014-10-05 21:33:44,145::task::1191::Storage.TaskManager.Task::(prepare) Task=`cb501390-cb62-4322-be31-6247184b7542`::finished: {'statuslist': [{'status': 0, 'id': u'af150e8e-4d74-4810-b8bf-0c6a19637dbf'}, {'status': 0, 'id': u'e8d58651-6414-4fc2-b281-2141cc97829b'}, {'status': 0, 'id': u'8f399b5f-1237-479e-82e6-eedb202b66fd'}]} >Thread-16::DEBUG::2014-10-05 21:33:44,145::task::595::Storage.TaskManager.Task::(_updateState) Task=`cb501390-cb62-4322-be31-6247184b7542`::moving from state preparing -> state finished >Thread-16::DEBUG::2014-10-05 21:33:44,145::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-16::DEBUG::2014-10-05 21:33:44,145::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-16::DEBUG::2014-10-05 21:33:44,145::task::993::Storage.TaskManager.Task::(_decref) Task=`cb501390-cb62-4322-be31-6247184b7542`::ref 0 aborting False >Thread-16::DEBUG::2014-10-05 21:33:44,146::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.connectStorageServer' in bridge with [{'status': 0, 'id': u'af150e8e-4d74-4810-b8bf-0c6a19637dbf'}, {'status': 0, 'id': u'e8d58651-6414-4fc2-b281-2141cc97829b'}, {'status': 0, 'id': u'8f399b5f-1237-479e-82e6-eedb202b66fd'}] >Thread-16::DEBUG::2014-10-05 21:33:44,146::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:33:44,257::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:33:44,259::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-17::DEBUG::2014-10-05 21:33:44,259::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.connect' in bridge with {u'masterVersion': 5, u'domainDict': {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': u'active', u'ff559f46-c495-4f6b-901c-2a624042a050': u'active'}, u'storagepoolID': u'b86b687a-d073-497a-ac8a-249025419a3e', u'scsiKey': u'b86b687a-d073-497a-ac8a-249025419a3e', u'masterSdUUID': u'ff559f46-c495-4f6b-901c-2a624042a050', u'hostID': 3} >Thread-17::DEBUG::2014-10-05 21:33:44,260::task::595::Storage.TaskManager.Task::(_updateState) Task=`a2258f12-6897-4dca-9939-2e3a181cb1ec`::moving from state init -> state preparing >Thread-17::INFO::2014-10-05 21:33:44,260::logUtils::44::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID=u'b86b687a-d073-497a-ac8a-249025419a3e', hostID=3, msdUUID=u'ff559f46-c495-4f6b-901c-2a624042a050', masterVersion=5, domainsMap={u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': u'active', u'ff559f46-c495-4f6b-901c-2a624042a050': u'active'}, options=None) >Thread-17::DEBUG::2014-10-05 21:33:44,261::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.HsmDomainMonitorLock`ReqID=`68412d70-60fe-4348-923a-872b970a03dc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1011' at 'connectStoragePool' >Thread-17::DEBUG::2014-10-05 21:33:44,261::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.HsmDomainMonitorLock' for lock type 'exclusive' >Thread-17::DEBUG::2014-10-05 21:33:44,261::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.HsmDomainMonitorLock' is free. Now locking as 'exclusive' (1 active user) >Thread-17::DEBUG::2014-10-05 21:33:44,261::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.HsmDomainMonitorLock`ReqID=`68412d70-60fe-4348-923a-872b970a03dc`::Granted request >Thread-17::DEBUG::2014-10-05 21:33:44,262::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.b86b687a-d073-497a-ac8a-249025419a3e`ReqID=`ea127862-f45f-494a-8ccd-f39eb7ad0e94`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1059' at '_connectStoragePool' >Thread-17::DEBUG::2014-10-05 21:33:44,262::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.b86b687a-d073-497a-ac8a-249025419a3e' for lock type 'exclusive' >Thread-17::DEBUG::2014-10-05 21:33:44,262::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.b86b687a-d073-497a-ac8a-249025419a3e' is free. Now locking as 'exclusive' (1 active user) >Thread-17::DEBUG::2014-10-05 21:33:44,262::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.b86b687a-d073-497a-ac8a-249025419a3e`ReqID=`ea127862-f45f-494a-8ccd-f39eb7ad0e94`::Granted request >Thread-17::DEBUG::2014-10-05 21:33:44,262::spbackends::485::Storage.StoragePoolMemoryBackend::(updateVersionAndDomains) updating domain version to 5 and domains map to {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': u'active', u'ff559f46-c495-4f6b-901c-2a624042a050': u'active'} >Thread-17::INFO::2014-10-05 21:33:44,263::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 5 and domains map {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': u'Active', u'ff559f46-c495-4f6b-901c-2a624042a050': u'Active'} >Thread-17::INFO::2014-10-05 21:33:44,263::sp::133::Storage.StoragePool::(setBackend) updating pool b86b687a-d073-497a-ac8a-249025419a3e backend from type NoneType instance 0x3a11372210 to type StoragePoolMemoryBackend instance 0x7f575c03c730 >Thread-17::INFO::2014-10-05 21:33:44,263::sp::631::Storage.StoragePool::(connect) Connect host #3 to the storage pool b86b687a-d073-497a-ac8a-249025419a3e with master domain: ff559f46-c495-4f6b-901c-2a624042a050 (ver = 5) >Thread-17::DEBUG::2014-10-05 21:33:44,263::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:44,263::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:44,263::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:44,263::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:44,263::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:44,264::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:44,264::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-17::DEBUG::2014-10-05 21:33:44,264::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-17::DEBUG::2014-10-05 21:33:44,264::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-17::DEBUG::2014-10-05 21:33:44,264::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-17::DEBUG::2014-10-05 21:33:44,264::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-17::DEBUG::2014-10-05 21:33:44,264::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-17::DEBUG::2014-10-05 21:33:44,433::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-17::DEBUG::2014-10-05 21:33:44,445::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-17::DEBUG::2014-10-05 21:33:44,572::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-17::DEBUG::2014-10-05 21:33:44,573::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:44,573::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:44,573::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:44,573::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:44,573::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:44,573::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:44,573::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-17::ERROR::2014-10-05 21:33:44,573::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain ff559f46-c495-4f6b-901c-2a624042a050 >Thread-17::ERROR::2014-10-05 21:33:44,574::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain ff559f46-c495-4f6b-901c-2a624042a050 >Thread-17::DEBUG::2014-10-05 21:33:44,574::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:44,576::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060001|/dev/mapper/1IET_00060002|/dev/mapper/1IET_00060003|/dev/mapper/1IET_00060004|/dev/mapper/1IET_00060005|/dev/mapper/1IET_00060006|/dev/mapper/1IET_00060007|/dev/mapper/1IET_00060008|/dev/mapper/1IET_00060009|/dev/mapper/1IET_0006000a|/dev/mapper/3600144f09dbd05000000522dde040011|/dev/mapper/3600144f09dbd05000000522dde050012|/dev/mapper/3600144f09dbd05000000522dde070013|/dev/mapper/3600144f09dbd05000000522dde090014|/dev/mapper/3600144f09dbd05000000522dde0a0015|/dev/mapper/3600144f09dbd05000000522dde0c0016|/dev/mapper/3600144f09dbd05000000522dde0d0017|/dev/mapper/3600144f09dbd05000000522dde0f0018|/dev/mapper/3600144f09dbd05000000522dde100019|/dev/mapper/3600144f09dbd05000000522dde12001a|/dev/mapper/3600144f09dbd0500000053a84cba0002|/dev/mapper/3600144f09dbd0500000053a84e730003|/dev/mapper/3600144f09dbd0500000053a84f4d0004|/dev/mapper/3600144f09dbd0500000053a8503b0005|/dev/mapper/3600144f09dbd0500000053a850500006|/dev/mapper/3600144f09dbd0500000053a8505b0007|/dev/mapper/3600144f09dbd0500000053a850640008|/dev/mapper/3600144f09dbd0500000053a8506c0009|/dev/mapper/3600144f09dbd0500000053a85076000a|/dev/mapper/3600144f09dbd0500000053a8507f000b|/dev/mapper/3600144f09dbd0500000053a853ed000c|/dev/mapper/3600144f09dbd0500000053a85401000d|/dev/mapper/3600144f09dbd0500000053a85409000e|/dev/mapper/3600144f09dbd0500000053a85410000f|/dev/mapper/3600144f09dbd0500000053a854180010|/dev/mapper/3600144f09dbd0500000053a854250011|/dev/mapper/3600144f09dbd0500000053a8542d0012|/dev/mapper/3600144f09dbd0500000053a8543e0013|/dev/mapper/3600144f09dbd0500000053a854450014|/dev/mapper/3600144f09dbd0500000053a8544d0015|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name ff559f46-c495-4f6b-901c-2a624042a050 (cwd None) >Thread-17::DEBUG::2014-10-05 21:33:44,794::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-17::DEBUG::2014-10-05 21:33:44,795::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:44,795::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-17::DEBUG::2014-10-05 21:33:44,795::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060007|/dev/mapper/1IET_00060009|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --available y ff559f46-c495-4f6b-901c-2a624042a050/metadata (cwd None) >Thread-17::DEBUG::2014-10-05 21:33:44,933::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-17::DEBUG::2014-10-05 21:33:44,933::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:44,933::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:44,933::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata count=1 (cwd None) >Thread-17::DEBUG::2014-10-05 21:33:44,937::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000627177 s, 3.3 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-10-05 21:33:44,937::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000627177 s, 3.3 MB/s'], size: 2048 >Thread-17::DEBUG::2014-10-05 21:33:44,938::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-17::DEBUG::2014-10-05 21:33:44,938::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-17::DEBUG::2014-10-05 21:33:44,938::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-17::DEBUG::2014-10-05 21:33:44,938::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:44,938::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:44,938::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:44,938::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:44,939::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:44,939::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060001|/dev/mapper/1IET_00060002|/dev/mapper/1IET_00060003|/dev/mapper/1IET_00060004|/dev/mapper/1IET_00060005|/dev/mapper/1IET_00060006|/dev/mapper/1IET_00060007|/dev/mapper/1IET_00060008|/dev/mapper/1IET_00060009|/dev/mapper/1IET_0006000a|/dev/mapper/3600144f09dbd05000000522dde040011|/dev/mapper/3600144f09dbd05000000522dde050012|/dev/mapper/3600144f09dbd05000000522dde070013|/dev/mapper/3600144f09dbd05000000522dde090014|/dev/mapper/3600144f09dbd05000000522dde0a0015|/dev/mapper/3600144f09dbd05000000522dde0c0016|/dev/mapper/3600144f09dbd05000000522dde0d0017|/dev/mapper/3600144f09dbd05000000522dde0f0018|/dev/mapper/3600144f09dbd05000000522dde100019|/dev/mapper/3600144f09dbd05000000522dde12001a|/dev/mapper/3600144f09dbd0500000053a84cba0002|/dev/mapper/3600144f09dbd0500000053a84e730003|/dev/mapper/3600144f09dbd0500000053a84f4d0004|/dev/mapper/3600144f09dbd0500000053a8503b0005|/dev/mapper/3600144f09dbd0500000053a850500006|/dev/mapper/3600144f09dbd0500000053a8505b0007|/dev/mapper/3600144f09dbd0500000053a850640008|/dev/mapper/3600144f09dbd0500000053a8506c0009|/dev/mapper/3600144f09dbd0500000053a85076000a|/dev/mapper/3600144f09dbd0500000053a8507f000b|/dev/mapper/3600144f09dbd0500000053a853ed000c|/dev/mapper/3600144f09dbd0500000053a85401000d|/dev/mapper/3600144f09dbd0500000053a85409000e|/dev/mapper/3600144f09dbd0500000053a85410000f|/dev/mapper/3600144f09dbd0500000053a854180010|/dev/mapper/3600144f09dbd0500000053a854250011|/dev/mapper/3600144f09dbd0500000053a8542d0012|/dev/mapper/3600144f09dbd0500000053a8543e0013|/dev/mapper/3600144f09dbd0500000053a854450014|/dev/mapper/3600144f09dbd0500000053a8544d0015|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name ff559f46-c495-4f6b-901c-2a624042a050 (cwd None) >Thread-17::DEBUG::2014-10-05 21:33:45,152::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-17::DEBUG::2014-10-05 21:33:45,152::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:45,152::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'VGUUID=1B1oT5-JcGj-Ru1I-v3Ni-yden-Du9b-I9fSu8', 'CLASS=Data', 'VERSION=3', 'TYPE=ISCSI', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'LOCKPOLICY=', 'SDUUID=ff559f46-c495-4f6b-901c-2a624042a050', 'PHYBLKSIZE=512', u'PV0=pv:1IET_00060007,uuid:QfuzQ5-Ml7u-kFUc-gdmr-rfmS-Aefp-3HODNb,pestart:0,pecount:157,mapoffset:0', 'POOL_UUID=b86b687a-d073-497a-ac8a-249025419a3e', 'ROLE=Master', u'PV1=pv:1IET_00060009,uuid:pGRciD-wcLp-X1ME-MijE-P1B9-7XWo-ycTXdG,pestart:0,pecount:157,mapoffset:157', 'DESCRIPTION=jumbo-el7', '_SHA_CKSUM=1695e16fba6531db32e52743127e50d763522d29'] >Thread-17::DEBUG::2014-10-05 21:33:45,153::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060007|/dev/mapper/1IET_00060009|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --available y ff559f46-c495-4f6b-901c-2a624042a050/ids (cwd None) >Thread-17::DEBUG::2014-10-05 21:33:45,298::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-17::DEBUG::2014-10-05 21:33:45,298::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:45,298::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:45,298::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060007|/dev/mapper/1IET_00060009|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --available y ff559f46-c495-4f6b-901c-2a624042a050/leases (cwd None) >Thread-17::DEBUG::2014-10-05 21:33:45,437::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-17::DEBUG::2014-10-05 21:33:45,437::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:45,437::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:45,437::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060007|/dev/mapper/1IET_00060009|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --available y ff559f46-c495-4f6b-901c-2a624042a050/inbox ff559f46-c495-4f6b-901c-2a624042a050/outbox ff559f46-c495-4f6b-901c-2a624042a050/master (cwd None) >Thread-17::DEBUG::2014-10-05 21:33:45,575::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-17::DEBUG::2014-10-05 21:33:45,575::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:45,575::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:45,576::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'ff559f46-c495-4f6b-901c-2a624042a050_imageNS' >Thread-17::DEBUG::2014-10-05 21:33:45,576::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'ff559f46-c495-4f6b-901c-2a624042a050_volumeNS' >Thread-17::DEBUG::2014-10-05 21:33:45,576::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'ff559f46-c495-4f6b-901c-2a624042a050_lvmActivationNS' >Thread-17::DEBUG::2014-10-05 21:33:45,577::sp::1390::Storage.StoragePool::(setMasterDomain) Master domain ff559f46-c495-4f6b-901c-2a624042a050 verified, version 5 >Thread-17::DEBUG::2014-10-05 21:33:45,577::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-17::DEBUG::2014-10-05 21:33:45,577::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-17::INFO::2014-10-05 21:33:45,577::domainMonitor::100::Storage.DomainMonitor::(startMonitoring) Start monitoring c73a3ced-4b41-4b4e-93f7-494d63e7e7ae >Thread-18::DEBUG::2014-10-05 21:33:45,577::domainMonitor::182::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for c73a3ced-4b41-4b4e-93f7-494d63e7e7ae >Thread-17::DEBUG::2014-10-05 21:33:45,577::sp::1413::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `b86b687a-d073-497a-ac8a-249025419a3e` started monitoring domain `c73a3ced-4b41-4b4e-93f7-494d63e7e7ae` >Thread-18::ERROR::2014-10-05 21:33:45,578::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain c73a3ced-4b41-4b4e-93f7-494d63e7e7ae >Thread-17::INFO::2014-10-05 21:33:45,578::domainMonitor::100::Storage.DomainMonitor::(startMonitoring) Start monitoring ff559f46-c495-4f6b-901c-2a624042a050 >Thread-18::ERROR::2014-10-05 21:33:45,578::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain c73a3ced-4b41-4b4e-93f7-494d63e7e7ae >Thread-19::DEBUG::2014-10-05 21:33:45,579::domainMonitor::182::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for ff559f46-c495-4f6b-901c-2a624042a050 >Thread-17::DEBUG::2014-10-05 21:33:45,579::sp::1413::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `b86b687a-d073-497a-ac8a-249025419a3e` started monitoring domain `ff559f46-c495-4f6b-901c-2a624042a050` >Thread-18::DEBUG::2014-10-05 21:33:45,580::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-17::DEBUG::2014-10-05 21:33:45,581::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-18::DEBUG::2014-10-05 21:33:45,581::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060001|/dev/mapper/1IET_00060002|/dev/mapper/1IET_00060003|/dev/mapper/1IET_00060004|/dev/mapper/1IET_00060005|/dev/mapper/1IET_00060006|/dev/mapper/1IET_00060007|/dev/mapper/1IET_00060008|/dev/mapper/1IET_00060009|/dev/mapper/1IET_0006000a|/dev/mapper/3600144f09dbd05000000522dde040011|/dev/mapper/3600144f09dbd05000000522dde050012|/dev/mapper/3600144f09dbd05000000522dde070013|/dev/mapper/3600144f09dbd05000000522dde090014|/dev/mapper/3600144f09dbd05000000522dde0a0015|/dev/mapper/3600144f09dbd05000000522dde0c0016|/dev/mapper/3600144f09dbd05000000522dde0d0017|/dev/mapper/3600144f09dbd05000000522dde0f0018|/dev/mapper/3600144f09dbd05000000522dde100019|/dev/mapper/3600144f09dbd05000000522dde12001a|/dev/mapper/3600144f09dbd0500000053a84cba0002|/dev/mapper/3600144f09dbd0500000053a84e730003|/dev/mapper/3600144f09dbd0500000053a84f4d0004|/dev/mapper/3600144f09dbd0500000053a8503b0005|/dev/mapper/3600144f09dbd0500000053a850500006|/dev/mapper/3600144f09dbd0500000053a8505b0007|/dev/mapper/3600144f09dbd0500000053a850640008|/dev/mapper/3600144f09dbd0500000053a8506c0009|/dev/mapper/3600144f09dbd0500000053a85076000a|/dev/mapper/3600144f09dbd0500000053a8507f000b|/dev/mapper/3600144f09dbd0500000053a853ed000c|/dev/mapper/3600144f09dbd0500000053a85401000d|/dev/mapper/3600144f09dbd0500000053a85409000e|/dev/mapper/3600144f09dbd0500000053a85410000f|/dev/mapper/3600144f09dbd0500000053a854180010|/dev/mapper/3600144f09dbd0500000053a854250011|/dev/mapper/3600144f09dbd0500000053a8542d0012|/dev/mapper/3600144f09dbd0500000053a8543e0013|/dev/mapper/3600144f09dbd0500000053a854450014|/dev/mapper/3600144f09dbd0500000053a8544d0015|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name c73a3ced-4b41-4b4e-93f7-494d63e7e7ae (cwd None) >Thread-17::DEBUG::2014-10-05 21:33:45,582::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e >Thread-19::DEBUG::2014-10-05 21:33:45,586::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgck --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060007|/dev/mapper/1IET_00060009|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' ff559f46-c495-4f6b-901c-2a624042a050 (cwd None) >Thread-17::DEBUG::2014-10-05 21:33:45,598::lvm::365::Storage.OperationMutex::(_reloadvgs) Got the operational mutex >Thread-17::DEBUG::2014-10-05 21:33:45,599::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060001|/dev/mapper/1IET_00060002|/dev/mapper/1IET_00060003|/dev/mapper/1IET_00060004|/dev/mapper/1IET_00060005|/dev/mapper/1IET_00060006|/dev/mapper/1IET_00060007|/dev/mapper/1IET_00060008|/dev/mapper/1IET_00060009|/dev/mapper/1IET_0006000a|/dev/mapper/3600144f09dbd05000000522dde040011|/dev/mapper/3600144f09dbd05000000522dde050012|/dev/mapper/3600144f09dbd05000000522dde070013|/dev/mapper/3600144f09dbd05000000522dde090014|/dev/mapper/3600144f09dbd05000000522dde0a0015|/dev/mapper/3600144f09dbd05000000522dde0c0016|/dev/mapper/3600144f09dbd05000000522dde0d0017|/dev/mapper/3600144f09dbd05000000522dde0f0018|/dev/mapper/3600144f09dbd05000000522dde100019|/dev/mapper/3600144f09dbd05000000522dde12001a|/dev/mapper/3600144f09dbd0500000053a84cba0002|/dev/mapper/3600144f09dbd0500000053a84e730003|/dev/mapper/3600144f09dbd0500000053a84f4d0004|/dev/mapper/3600144f09dbd0500000053a8503b0005|/dev/mapper/3600144f09dbd0500000053a850500006|/dev/mapper/3600144f09dbd0500000053a8505b0007|/dev/mapper/3600144f09dbd0500000053a850640008|/dev/mapper/3600144f09dbd0500000053a8506c0009|/dev/mapper/3600144f09dbd0500000053a85076000a|/dev/mapper/3600144f09dbd0500000053a8507f000b|/dev/mapper/3600144f09dbd0500000053a853ed000c|/dev/mapper/3600144f09dbd0500000053a85401000d|/dev/mapper/3600144f09dbd0500000053a85409000e|/dev/mapper/3600144f09dbd0500000053a85410000f|/dev/mapper/3600144f09dbd0500000053a854180010|/dev/mapper/3600144f09dbd0500000053a854250011|/dev/mapper/3600144f09dbd0500000053a8542d0012|/dev/mapper/3600144f09dbd0500000053a8543e0013|/dev/mapper/3600144f09dbd0500000053a854450014|/dev/mapper/3600144f09dbd0500000053a8544d0015|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name c73a3ced-4b41-4b4e-93f7-494d63e7e7ae (cwd None) >Thread-19::DEBUG::2014-10-05 21:33:45,726::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-19::DEBUG::2014-10-05 21:33:45,726::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-19::DEBUG::2014-10-05 21:33:45,730::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000356436 s, 11.5 MB/s\n'; <rc> = 0 >Thread-19::DEBUG::2014-10-05 21:33:45,734::domainMonitor::264::Storage.DomainMonitorThread::(_monitorDomain) Domain ff559f46-c495-4f6b-901c-2a624042a050 changed its status to Valid >Thread-20::DEBUG::2014-10-05 21:33:45,735::misc::821::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Emitting event >Thread-19::INFO::2014-10-05 21:33:45,735::clusterlock::222::Storage.SANLock::(acquireHostId) Acquiring host id for domain ff559f46-c495-4f6b-901c-2a624042a050 (id: 3) >Thread-20::DEBUG::2014-10-05 21:33:45,735::misc::831::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `contEIOVms` >Thread-19::DEBUG::2014-10-05 21:33:45,736::clusterlock::240::Storage.SANLock::(acquireHostId) Host id for domain ff559f46-c495-4f6b-901c-2a624042a050 successfully acquired (id: 3) >Thread-20::DEBUG::2014-10-05 21:33:45,736::misc::841::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Event emitted >Thread-21::INFO::2014-10-05 21:33:45,737::clientIF::136::vds::(contEIOVms) vmContainerLock acquired >Thread-17::DEBUG::2014-10-05 21:33:45,817::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-17::DEBUG::2014-10-05 21:33:45,818::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/dom_md >Thread-17::WARNING::2014-10-05 21:33:45,818::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/dom_md already exists >Thread-17::DEBUG::2014-10-05 21:33:45,818::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/images >Thread-17::WARNING::2014-10-05 21:33:45,818::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/images already exists >Thread-17::INFO::2014-10-05 21:33:45,819::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae to /rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae >Thread-17::DEBUG::2014-10-05 21:33:45,819::sp::1137::Storage.StoragePool::(_linkStorageDomain) Creating symlink from /rhev/data-center/mnt/blockSD/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae to /rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae >Thread-17::DEBUG::2014-10-05 21:33:45,819::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050/images >Thread-17::WARNING::2014-10-05 21:33:45,819::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050/images already exists >Thread-17::DEBUG::2014-10-05 21:33:45,819::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050/dom_md >Thread-17::WARNING::2014-10-05 21:33:45,819::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050/dom_md already exists >Thread-17::INFO::2014-10-05 21:33:45,820::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050 to /rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/ff559f46-c495-4f6b-901c-2a624042a050 >Thread-17::DEBUG::2014-10-05 21:33:45,820::sp::1137::Storage.StoragePool::(_linkStorageDomain) Creating symlink from /rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050 to /rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/ff559f46-c495-4f6b-901c-2a624042a050 >Thread-17::INFO::2014-10-05 21:33:45,820::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050 to /rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/mastersd >Thread-17::DEBUG::2014-10-05 21:33:45,820::sp::1137::Storage.StoragePool::(_linkStorageDomain) Creating symlink from /rhev/data-center/mnt/blockSD/ff559f46-c495-4f6b-901c-2a624042a050 to /rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/mastersd >Thread-17::DEBUG::2014-10-05 21:33:45,821::threadPool::35::Storage.ThreadPool::(__init__) Enter - numThreads: 5, waitTimeout: 3, maxTasks: 500 >Thread-17::DEBUG::2014-10-05 21:33:45,822::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /bin/dd if=/rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/mastersd/dom_md/outbox iflag=direct,fullblock bs=512 count=8 skip=24 (cwd None) >Thread-17::DEBUG::2014-10-05 21:33:45,829::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.00313734 s, 1.3 MB/s\n'; <rc> = 0 >Thread-17::INFO::2014-10-05 21:33:45,829::storage_mailbox::385::Storage.MailBox.HsmMailMonitor::(_sendMail) HSM_MailMonitor sending mail to SPM - ['/bin/dd', 'of=/rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/mastersd/dom_md/inbox', 'iflag=fullblock', 'oflag=direct', 'conv=notrunc', 'bs=512', 'seek=24'] >Thread-17::DEBUG::2014-10-05 21:33:45,829::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /bin/dd of=/rhev/data-center/b86b687a-d073-497a-ac8a-249025419a3e/mastersd/dom_md/inbox iflag=fullblock oflag=direct conv=notrunc bs=512 seek=24 (cwd None) >Thread-17::DEBUG::2014-10-05 21:33:45,836::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.00276373 s, 1.5 MB/s\n'; <rc> = 0 >Thread-17::DEBUG::2014-10-05 21:33:45,836::storage_mailbox::207::Storage.Mailbox.HSM::(__init__) HSM_MailboxMonitor created for pool b86b687a-d073-497a-ac8a-249025419a3e >Thread-17::DEBUG::2014-10-05 21:33:45,837::sp::462::Storage.StoragePool::(__createMailboxMonitor) HSM mailbox ready for pool b86b687a-d073-497a-ac8a-249025419a3e on master domain ff559f46-c495-4f6b-901c-2a624042a050 >Thread-17::DEBUG::2014-10-05 21:33:45,837::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.b86b687a-d073-497a-ac8a-249025419a3e' >Thread-17::DEBUG::2014-10-05 21:33:45,837::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.b86b687a-d073-497a-ac8a-249025419a3e' (0 active users) >Thread-17::DEBUG::2014-10-05 21:33:45,837::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.b86b687a-d073-497a-ac8a-249025419a3e' is free, finding out if anyone is waiting for it. >Thread-17::DEBUG::2014-10-05 21:33:45,838::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.b86b687a-d073-497a-ac8a-249025419a3e', Clearing records. >Thread-17::DEBUG::2014-10-05 21:33:45,838::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.HsmDomainMonitorLock' >Thread-17::DEBUG::2014-10-05 21:33:45,838::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.HsmDomainMonitorLock' (0 active users) >Thread-17::DEBUG::2014-10-05 21:33:45,838::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.HsmDomainMonitorLock' is free, finding out if anyone is waiting for it. >Thread-17::DEBUG::2014-10-05 21:33:45,838::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.HsmDomainMonitorLock', Clearing records. >Thread-17::INFO::2014-10-05 21:33:45,839::logUtils::47::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True >Thread-17::DEBUG::2014-10-05 21:33:45,839::task::1191::Storage.TaskManager.Task::(prepare) Task=`a2258f12-6897-4dca-9939-2e3a181cb1ec`::finished: True >Thread-17::DEBUG::2014-10-05 21:33:45,839::task::595::Storage.TaskManager.Task::(_updateState) Task=`a2258f12-6897-4dca-9939-2e3a181cb1ec`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-10-05 21:33:45,839::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-10-05 21:33:45,839::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-10-05 21:33:45,840::task::993::Storage.TaskManager.Task::(_decref) Task=`a2258f12-6897-4dca-9939-2e3a181cb1ec`::ref 0 aborting False >Thread-17::DEBUG::2014-10-05 21:33:45,840::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.connect' in bridge with True >Thread-17::DEBUG::2014-10-05 21:33:45,840::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-18::DEBUG::2014-10-05 21:33:45,847::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:33:45,847::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-18::DEBUG::2014-10-05 21:33:45,847::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-18::DEBUG::2014-10-05 21:33:45,847::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/3600144f09dbd0500000053a84e730003|/dev/mapper/3600144f09dbd0500000053a85076000a|/dev/mapper/3600144f09dbd0500000053a853ed000c|/dev/mapper/3600144f09dbd0500000053a85401000d|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --available y c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata (cwd None) >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:33:45,928::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:33:45,929::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-28::DEBUG::2014-10-05 21:33:45,931::task::595::Storage.TaskManager.Task::(_updateState) Task=`354f28ed-9efd-4190-8ee4-a961cb0fa6cd`::moving from state init -> state preparing >Thread-28::INFO::2014-10-05 21:33:45,932::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-28::INFO::2014-10-05 21:33:45,932::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': -1, 'acquired': False, 'delay': '0', 'lastCheck': '0.4', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000356436', 'lastCheck': '0.2', 'valid': True}} >Thread-28::DEBUG::2014-10-05 21:33:45,932::task::1191::Storage.TaskManager.Task::(prepare) Task=`354f28ed-9efd-4190-8ee4-a961cb0fa6cd`::finished: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': -1, 'acquired': False, 'delay': '0', 'lastCheck': '0.4', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000356436', 'lastCheck': '0.2', 'valid': True}} >Thread-28::DEBUG::2014-10-05 21:33:45,932::task::595::Storage.TaskManager.Task::(_updateState) Task=`354f28ed-9efd-4190-8ee4-a961cb0fa6cd`::moving from state preparing -> state finished >Thread-28::DEBUG::2014-10-05 21:33:45,932::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-28::DEBUG::2014-10-05 21:33:45,932::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-28::DEBUG::2014-10-05 21:33:45,932::task::993::Storage.TaskManager.Task::(_decref) Task=`354f28ed-9efd-4190-8ee4-a961cb0fa6cd`::ref 0 aborting False >Thread-28::DEBUG::2014-10-05 21:33:45,934::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-18::DEBUG::2014-10-05 21:33:46,000::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:33:46,000::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-18::DEBUG::2014-10-05 21:33:46,000::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-18::DEBUG::2014-10-05 21:33:46,000::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata count=1 (cwd None) >Thread-18::DEBUG::2014-10-05 21:33:46,005::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000514518 s, 4.0 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:33:46,005::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000514518 s, 4.0 MB/s'], size: 2048 >Thread-18::DEBUG::2014-10-05 21:33:46,005::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-18::DEBUG::2014-10-05 21:33:46,006::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-18::DEBUG::2014-10-05 21:33:46,006::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-18::DEBUG::2014-10-05 21:33:46,006::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-18::DEBUG::2014-10-05 21:33:46,006::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-18::DEBUG::2014-10-05 21:33:46,006::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-18::DEBUG::2014-10-05 21:33:46,006::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-18::DEBUG::2014-10-05 21:33:46,006::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-18::DEBUG::2014-10-05 21:33:46,007::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1IET_00060001|/dev/mapper/1IET_00060002|/dev/mapper/1IET_00060003|/dev/mapper/1IET_00060004|/dev/mapper/1IET_00060005|/dev/mapper/1IET_00060006|/dev/mapper/1IET_00060007|/dev/mapper/1IET_00060008|/dev/mapper/1IET_00060009|/dev/mapper/1IET_0006000a|/dev/mapper/3600144f09dbd05000000522dde040011|/dev/mapper/3600144f09dbd05000000522dde050012|/dev/mapper/3600144f09dbd05000000522dde070013|/dev/mapper/3600144f09dbd05000000522dde090014|/dev/mapper/3600144f09dbd05000000522dde0a0015|/dev/mapper/3600144f09dbd05000000522dde0c0016|/dev/mapper/3600144f09dbd05000000522dde0d0017|/dev/mapper/3600144f09dbd05000000522dde0f0018|/dev/mapper/3600144f09dbd05000000522dde100019|/dev/mapper/3600144f09dbd05000000522dde12001a|/dev/mapper/3600144f09dbd0500000053a84cba0002|/dev/mapper/3600144f09dbd0500000053a84e730003|/dev/mapper/3600144f09dbd0500000053a84f4d0004|/dev/mapper/3600144f09dbd0500000053a8503b0005|/dev/mapper/3600144f09dbd0500000053a850500006|/dev/mapper/3600144f09dbd0500000053a8505b0007|/dev/mapper/3600144f09dbd0500000053a850640008|/dev/mapper/3600144f09dbd0500000053a8506c0009|/dev/mapper/3600144f09dbd0500000053a85076000a|/dev/mapper/3600144f09dbd0500000053a8507f000b|/dev/mapper/3600144f09dbd0500000053a853ed000c|/dev/mapper/3600144f09dbd0500000053a85401000d|/dev/mapper/3600144f09dbd0500000053a85409000e|/dev/mapper/3600144f09dbd0500000053a85410000f|/dev/mapper/3600144f09dbd0500000053a854180010|/dev/mapper/3600144f09dbd0500000053a854250011|/dev/mapper/3600144f09dbd0500000053a8542d0012|/dev/mapper/3600144f09dbd0500000053a8543e0013|/dev/mapper/3600144f09dbd0500000053a854450014|/dev/mapper/3600144f09dbd0500000053a8544d0015|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name c73a3ced-4b41-4b4e-93f7-494d63e7e7ae (cwd None) >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:33:46,027::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:33:46,029::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-29::DEBUG::2014-10-05 21:33:46,029::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.setMOMPolicyParameters' in bridge with {u'key_value_store': {u'balloonEnabled': True, u'ksmEnabled': True}} >Thread-29::DEBUG::2014-10-05 21:33:46,070::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.setMOMPolicyParameters' in bridge with True >Thread-29::DEBUG::2014-10-05 21:33:46,071::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-18::DEBUG::2014-10-05 21:33:46,219::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:33:46,219::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-18::DEBUG::2014-10-05 21:33:46,219::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', u'PV1=pv:3600144f09dbd0500000053a84e730003,uuid:5zWyZx-cpAZ-tMON-3wa5-s6pN-Xj7N-b3fPs5,pestart:0,pecount:77,mapoffset:77', 'DESCRIPTION=sasha-el7', 'VERSION=3', u'PV2=pv:3600144f09dbd0500000053a853ed000c,uuid:c2LMVb-PXdj-XpTE-4f06-2rrh-TdAG-zzehcn,pestart:0,pecount:77,mapoffset:154', 'SDUUID=c73a3ced-4b41-4b4e-93f7-494d63e7e7ae', 'LOGBLKSIZE=512', 'TYPE=ISCSI', u'PV3=pv:3600144f09dbd0500000053a85401000d,uuid:1kMage-eAId-qy7G-oIqH-tfbH-ScLD-gqv7JM,pestart:0,pecount:77,mapoffset:231', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'PHYBLKSIZE=512', 'VGUUID=pF71zc-KPBX-BMKm-7KHK-KihY-ysp4-6DZTpA', 'CLASS=Data', 'LOCKPOLICY=', 'ROLE=Regular', u'PV0=pv:3600144f09dbd0500000053a85076000a,uuid:Ka3ewV-ddPQ-boL6-sDJT-xcFb-YBkc-9HeKFq,pestart:0,pecount:77,mapoffset:0', 'POOL_UUID=b86b687a-d073-497a-ac8a-249025419a3e', '_SHA_CKSUM=40993341560adfe664d3c1ab0c6b62068addc74c'] >Thread-18::DEBUG::2014-10-05 21:33:46,220::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/3600144f09dbd0500000053a84e730003|/dev/mapper/3600144f09dbd0500000053a85076000a|/dev/mapper/3600144f09dbd0500000053a853ed000c|/dev/mapper/3600144f09dbd0500000053a85401000d|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --available y c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/ids (cwd None) >Thread-18::DEBUG::2014-10-05 21:33:46,353::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:33:46,354::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-18::DEBUG::2014-10-05 21:33:46,354::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-18::DEBUG::2014-10-05 21:33:46,354::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/3600144f09dbd0500000053a84e730003|/dev/mapper/3600144f09dbd0500000053a85076000a|/dev/mapper/3600144f09dbd0500000053a853ed000c|/dev/mapper/3600144f09dbd0500000053a85401000d|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --available y c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/leases (cwd None) >Thread-18::DEBUG::2014-10-05 21:33:46,487::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:33:46,488::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-18::DEBUG::2014-10-05 21:33:46,488::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-18::DEBUG::2014-10-05 21:33:46,488::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/3600144f09dbd0500000053a84e730003|/dev/mapper/3600144f09dbd0500000053a85076000a|/dev/mapper/3600144f09dbd0500000053a853ed000c|/dev/mapper/3600144f09dbd0500000053a85401000d|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --available y c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/inbox c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/outbox c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/master (cwd None) >Thread-18::DEBUG::2014-10-05 21:33:46,623::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:33:46,623::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-18::DEBUG::2014-10-05 21:33:46,623::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-18::DEBUG::2014-10-05 21:33:46,624::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae_imageNS' >Thread-18::DEBUG::2014-10-05 21:33:46,624::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae_volumeNS' >Thread-18::DEBUG::2014-10-05 21:33:46,624::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae_lvmActivationNS' >Thread-18::DEBUG::2014-10-05 21:33:46,624::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgck --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/3600144f09dbd0500000053a84e730003|/dev/mapper/3600144f09dbd0500000053a85076000a|/dev/mapper/3600144f09dbd0500000053a853ed000c|/dev/mapper/3600144f09dbd0500000053a85401000d|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' c73a3ced-4b41-4b4e-93f7-494d63e7e7ae (cwd None) >Thread-18::DEBUG::2014-10-05 21:33:46,719::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:33:46,719::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-18::DEBUG::2014-10-05 21:33:46,723::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000510447 s, 8.0 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:33:46,724::domainMonitor::264::Storage.DomainMonitorThread::(_monitorDomain) Domain c73a3ced-4b41-4b4e-93f7-494d63e7e7ae changed its status to Valid >Thread-30::DEBUG::2014-10-05 21:33:46,725::misc::821::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Emitting event >Thread-18::INFO::2014-10-05 21:33:46,725::clusterlock::222::Storage.SANLock::(acquireHostId) Acquiring host id for domain c73a3ced-4b41-4b4e-93f7-494d63e7e7ae (id: 3) >Thread-30::DEBUG::2014-10-05 21:33:46,726::misc::831::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `contEIOVms` >Thread-18::DEBUG::2014-10-05 21:33:46,726::clusterlock::240::Storage.SANLock::(acquireHostId) Host id for domain c73a3ced-4b41-4b4e-93f7-494d63e7e7ae successfully acquired (id: 3) >Thread-30::DEBUG::2014-10-05 21:33:46,726::misc::841::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Event emitted >Thread-31::INFO::2014-10-05 21:33:46,727::clientIF::136::vds::(contEIOVms) vmContainerLock acquired >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:33:49,239::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:33:49,239::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-32::DEBUG::2014-10-05 21:33:49,240::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:33:52,328::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:33:52,329::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-33::DEBUG::2014-10-05 21:33:52,331::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:33:55,421::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:33:55,423::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-34::DEBUG::2014-10-05 21:33:55,425::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-19::DEBUG::2014-10-05 21:33:55,737::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-19::DEBUG::2014-10-05 21:33:55,741::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000710676 s, 5.8 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:33:56,728::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-18::DEBUG::2014-10-05 21:33:56,732::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000493771 s, 8.3 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:33:58,518::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:33:58,523::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-35::DEBUG::2014-10-05 21:33:58,524::task::595::Storage.TaskManager.Task::(_updateState) Task=`99242e88-1651-4e35-97c2-9e61f520d855`::moving from state init -> state preparing >Thread-35::INFO::2014-10-05 21:33:58,524::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-35::INFO::2014-10-05 21:33:58,524::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000493771', 'lastCheck': '1.8', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000710676', 'lastCheck': '2.8', 'valid': True}} >Thread-35::DEBUG::2014-10-05 21:33:58,524::task::1191::Storage.TaskManager.Task::(prepare) Task=`99242e88-1651-4e35-97c2-9e61f520d855`::finished: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000493771', 'lastCheck': '1.8', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000710676', 'lastCheck': '2.8', 'valid': True}} >Thread-35::DEBUG::2014-10-05 21:33:58,525::task::595::Storage.TaskManager.Task::(_updateState) Task=`99242e88-1651-4e35-97c2-9e61f520d855`::moving from state preparing -> state finished >Thread-35::DEBUG::2014-10-05 21:33:58,525::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-35::DEBUG::2014-10-05 21:33:58,525::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-35::DEBUG::2014-10-05 21:33:58,525::task::993::Storage.TaskManager.Task::(_decref) Task=`99242e88-1651-4e35-97c2-9e61f520d855`::ref 0 aborting False >Thread-35::DEBUG::2014-10-05 21:33:58,526::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:33:58,621::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:33:58,622::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-36::DEBUG::2014-10-05 21:33:58,622::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:34:01,778::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:34:01,779::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-37::DEBUG::2014-10-05 21:34:01,780::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:34:04,870::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:34:04,871::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-38::DEBUG::2014-10-05 21:34:04,872::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-19::DEBUG::2014-10-05 21:34:05,744::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-19::DEBUG::2014-10-05 21:34:05,748::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000655251 s, 6.3 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:34:06,733::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-18::DEBUG::2014-10-05 21:34:06,737::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000676123 s, 6.1 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:34:07,960::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:34:07,961::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-39::DEBUG::2014-10-05 21:34:07,963::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:34:11,066::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:34:11,067::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-40::DEBUG::2014-10-05 21:34:11,068::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:34:14,156::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:34:14,156::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-41::DEBUG::2014-10-05 21:34:14,158::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb3092b8-228b-4a84-bc4e-51bf998ddd8e`::moving from state init -> state preparing >Thread-41::INFO::2014-10-05 21:34:14,158::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-41::INFO::2014-10-05 21:34:14,158::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000676123', 'lastCheck': '7.4', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000655251', 'lastCheck': '8.4', 'valid': True}} >Thread-41::DEBUG::2014-10-05 21:34:14,158::task::1191::Storage.TaskManager.Task::(prepare) Task=`bb3092b8-228b-4a84-bc4e-51bf998ddd8e`::finished: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000676123', 'lastCheck': '7.4', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000655251', 'lastCheck': '8.4', 'valid': True}} >Thread-41::DEBUG::2014-10-05 21:34:14,158::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb3092b8-228b-4a84-bc4e-51bf998ddd8e`::moving from state preparing -> state finished >Thread-41::DEBUG::2014-10-05 21:34:14,158::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-41::DEBUG::2014-10-05 21:34:14,158::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-41::DEBUG::2014-10-05 21:34:14,158::task::993::Storage.TaskManager.Task::(_decref) Task=`bb3092b8-228b-4a84-bc4e-51bf998ddd8e`::ref 0 aborting False >Thread-41::DEBUG::2014-10-05 21:34:14,160::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:34:14,256::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:34:14,257::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-42::DEBUG::2014-10-05 21:34:14,258::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-19::DEBUG::2014-10-05 21:34:15,754::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-19::DEBUG::2014-10-05 21:34:15,758::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000806287 s, 5.1 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:34:16,738::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-18::DEBUG::2014-10-05 21:34:16,742::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000550423 s, 7.4 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:34:17,400::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:34:17,401::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-43::DEBUG::2014-10-05 21:34:17,402::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:34:20,489::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:34:20,490::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-44::DEBUG::2014-10-05 21:34:20,490::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:34:23,577::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:34:23,578::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-45::DEBUG::2014-10-05 21:34:23,580::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-19::DEBUG::2014-10-05 21:34:25,759::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-19::DEBUG::2014-10-05 21:34:25,764::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000876025 s, 4.7 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:34:26,667::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:34:26,668::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-46::DEBUG::2014-10-05 21:34:26,670::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-18::DEBUG::2014-10-05 21:34:26,743::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-18::DEBUG::2014-10-05 21:34:26,747::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00065036 s, 6.3 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:34:29,758::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:34:29,759::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-47::DEBUG::2014-10-05 21:34:29,760::task::595::Storage.TaskManager.Task::(_updateState) Task=`95786752-af62-4c5b-8dd7-513af9e4ec8f`::moving from state init -> state preparing >Thread-47::INFO::2014-10-05 21:34:29,760::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-10-05 21:34:29,760::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00065036', 'lastCheck': '3.0', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000876025', 'lastCheck': '4.0', 'valid': True}} >Thread-47::DEBUG::2014-10-05 21:34:29,760::task::1191::Storage.TaskManager.Task::(prepare) Task=`95786752-af62-4c5b-8dd7-513af9e4ec8f`::finished: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00065036', 'lastCheck': '3.0', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000876025', 'lastCheck': '4.0', 'valid': True}} >Thread-47::DEBUG::2014-10-05 21:34:29,761::task::595::Storage.TaskManager.Task::(_updateState) Task=`95786752-af62-4c5b-8dd7-513af9e4ec8f`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-10-05 21:34:29,761::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-10-05 21:34:29,761::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-10-05 21:34:29,761::task::993::Storage.TaskManager.Task::(_decref) Task=`95786752-af62-4c5b-8dd7-513af9e4ec8f`::ref 0 aborting False >Thread-47::DEBUG::2014-10-05 21:34:29,762::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:34:29,851::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:34:29,852::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-48::DEBUG::2014-10-05 21:34:29,853::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:34:33,006::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:34:33,007::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-49::DEBUG::2014-10-05 21:34:33,008::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-19::DEBUG::2014-10-05 21:34:35,766::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-19::DEBUG::2014-10-05 21:34:35,771::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000735345 s, 5.6 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:34:36,098::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:34:36,099::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-50::DEBUG::2014-10-05 21:34:36,101::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-18::DEBUG::2014-10-05 21:34:36,748::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-18::DEBUG::2014-10-05 21:34:36,752::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000647269 s, 6.3 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:34:39,189::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:34:39,190::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-51::DEBUG::2014-10-05 21:34:39,192::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:34:42,285::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:34:42,286::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-52::DEBUG::2014-10-05 21:34:42,287::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:34:45,375::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:34:45,375::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-53::DEBUG::2014-10-05 21:34:45,376::task::595::Storage.TaskManager.Task::(_updateState) Task=`3fa2d8a2-0b55-4fd9-87e5-6894000db57e`::moving from state init -> state preparing >Thread-53::INFO::2014-10-05 21:34:45,376::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-53::INFO::2014-10-05 21:34:45,377::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000647269', 'lastCheck': '8.6', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000735345', 'lastCheck': '9.6', 'valid': True}} >Thread-53::DEBUG::2014-10-05 21:34:45,377::task::1191::Storage.TaskManager.Task::(prepare) Task=`3fa2d8a2-0b55-4fd9-87e5-6894000db57e`::finished: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000647269', 'lastCheck': '8.6', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000735345', 'lastCheck': '9.6', 'valid': True}} >Thread-53::DEBUG::2014-10-05 21:34:45,377::task::595::Storage.TaskManager.Task::(_updateState) Task=`3fa2d8a2-0b55-4fd9-87e5-6894000db57e`::moving from state preparing -> state finished >Thread-53::DEBUG::2014-10-05 21:34:45,377::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-53::DEBUG::2014-10-05 21:34:45,377::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-53::DEBUG::2014-10-05 21:34:45,377::task::993::Storage.TaskManager.Task::(_decref) Task=`3fa2d8a2-0b55-4fd9-87e5-6894000db57e`::ref 0 aborting False >Thread-53::DEBUG::2014-10-05 21:34:45,378::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:34:45,470::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:34:45,470::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-54::DEBUG::2014-10-05 21:34:45,471::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-19::DEBUG::2014-10-05 21:34:45,772::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-19::DEBUG::2014-10-05 21:34:45,777::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.0007277 s, 5.6 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:34:46,754::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-18::DEBUG::2014-10-05 21:34:46,758::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000648386 s, 6.3 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:34:48,272::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:34:48,273::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-55::DEBUG::2014-10-05 21:34:48,275::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:34:51,366::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:34:51,367::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-56::DEBUG::2014-10-05 21:34:51,368::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:34:54,461::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:34:54,463::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-57::DEBUG::2014-10-05 21:34:54,465::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-19::DEBUG::2014-10-05 21:34:55,779::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-19::DEBUG::2014-10-05 21:34:55,783::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000678324 s, 6.0 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:34:56,759::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-18::DEBUG::2014-10-05 21:34:56,763::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000589184 s, 7.0 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:34:57,553::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:34:57,554::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-58::DEBUG::2014-10-05 21:34:57,555::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:35:00,646::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:35:00,646::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-59::DEBUG::2014-10-05 21:35:00,647::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ce0409d-0d0e-42f8-ba75-dc9a3092136e`::moving from state init -> state preparing >Thread-59::INFO::2014-10-05 21:35:00,647::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-59::INFO::2014-10-05 21:35:00,647::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000589184', 'lastCheck': '3.9', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000678324', 'lastCheck': '4.9', 'valid': True}} >Thread-59::DEBUG::2014-10-05 21:35:00,648::task::1191::Storage.TaskManager.Task::(prepare) Task=`0ce0409d-0d0e-42f8-ba75-dc9a3092136e`::finished: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000589184', 'lastCheck': '3.9', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000678324', 'lastCheck': '4.9', 'valid': True}} >Thread-59::DEBUG::2014-10-05 21:35:00,648::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ce0409d-0d0e-42f8-ba75-dc9a3092136e`::moving from state preparing -> state finished >Thread-59::DEBUG::2014-10-05 21:35:00,648::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-59::DEBUG::2014-10-05 21:35:00,648::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-59::DEBUG::2014-10-05 21:35:00,648::task::993::Storage.TaskManager.Task::(_decref) Task=`0ce0409d-0d0e-42f8-ba75-dc9a3092136e`::ref 0 aborting False >Thread-59::DEBUG::2014-10-05 21:35:00,649::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:35:00,745::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:35:00,745::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-60::DEBUG::2014-10-05 21:35:00,746::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:35:03,939::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:35:03,940::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-61::DEBUG::2014-10-05 21:35:03,941::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-19::DEBUG::2014-10-05 21:35:05,785::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-19::DEBUG::2014-10-05 21:35:05,790::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000776892 s, 5.3 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:35:06,764::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-18::DEBUG::2014-10-05 21:35:06,768::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000601489 s, 6.8 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:35:07,029::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:35:07,030::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-62::DEBUG::2014-10-05 21:35:07,031::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:35:10,121::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:35:10,121::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-63::DEBUG::2014-10-05 21:35:10,122::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:35:13,208::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:35:13,209::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-64::DEBUG::2014-10-05 21:35:13,210::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-19::DEBUG::2014-10-05 21:35:15,792::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-19::DEBUG::2014-10-05 21:35:15,798::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000722478 s, 5.7 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:35:16,349::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:35:16,350::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-65::DEBUG::2014-10-05 21:35:16,352::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0eb8713-b664-4cd8-9cd2-5b315032d20c`::moving from state init -> state preparing >Thread-65::INFO::2014-10-05 21:35:16,352::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-65::INFO::2014-10-05 21:35:16,352::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000601489', 'lastCheck': '9.6', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000722478', 'lastCheck': '0.5', 'valid': True}} >Thread-65::DEBUG::2014-10-05 21:35:16,352::task::1191::Storage.TaskManager.Task::(prepare) Task=`d0eb8713-b664-4cd8-9cd2-5b315032d20c`::finished: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000601489', 'lastCheck': '9.6', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000722478', 'lastCheck': '0.5', 'valid': True}} >Thread-65::DEBUG::2014-10-05 21:35:16,352::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0eb8713-b664-4cd8-9cd2-5b315032d20c`::moving from state preparing -> state finished >Thread-65::DEBUG::2014-10-05 21:35:16,352::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-65::DEBUG::2014-10-05 21:35:16,352::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-65::DEBUG::2014-10-05 21:35:16,353::task::993::Storage.TaskManager.Task::(_decref) Task=`d0eb8713-b664-4cd8-9cd2-5b315032d20c`::ref 0 aborting False >Thread-65::DEBUG::2014-10-05 21:35:16,354::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:35:16,547::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:35:16,548::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-66::DEBUG::2014-10-05 21:35:16,549::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-18::DEBUG::2014-10-05 21:35:16,770::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-18::DEBUG::2014-10-05 21:35:16,773::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000559044 s, 7.3 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:35:19,700::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:35:19,701::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-67::DEBUG::2014-10-05 21:35:19,703::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:35:22,794::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:35:22,795::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-68::DEBUG::2014-10-05 21:35:22,796::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-19::DEBUG::2014-10-05 21:35:25,805::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-19::DEBUG::2014-10-05 21:35:25,809::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000626597 s, 6.5 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:35:25,885::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:35:25,886::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-69::DEBUG::2014-10-05 21:35:25,887::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-18::DEBUG::2014-10-05 21:35:26,775::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-18::DEBUG::2014-10-05 21:35:26,779::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000568807 s, 7.2 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:35:28,974::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:35:28,975::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-70::DEBUG::2014-10-05 21:35:28,976::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:35:32,063::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:35:32,064::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-71::DEBUG::2014-10-05 21:35:32,065::task::595::Storage.TaskManager.Task::(_updateState) Task=`e9bc7483-0eee-482c-83ba-fc9d90cb5f16`::moving from state init -> state preparing >Thread-71::INFO::2014-10-05 21:35:32,066::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-71::INFO::2014-10-05 21:35:32,066::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000568807', 'lastCheck': '5.3', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000626597', 'lastCheck': '6.3', 'valid': True}} >Thread-71::DEBUG::2014-10-05 21:35:32,066::task::1191::Storage.TaskManager.Task::(prepare) Task=`e9bc7483-0eee-482c-83ba-fc9d90cb5f16`::finished: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000568807', 'lastCheck': '5.3', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000626597', 'lastCheck': '6.3', 'valid': True}} >Thread-71::DEBUG::2014-10-05 21:35:32,066::task::595::Storage.TaskManager.Task::(_updateState) Task=`e9bc7483-0eee-482c-83ba-fc9d90cb5f16`::moving from state preparing -> state finished >Thread-71::DEBUG::2014-10-05 21:35:32,066::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-71::DEBUG::2014-10-05 21:35:32,066::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-71::DEBUG::2014-10-05 21:35:32,066::task::993::Storage.TaskManager.Task::(_decref) Task=`e9bc7483-0eee-482c-83ba-fc9d90cb5f16`::ref 0 aborting False >Thread-71::DEBUG::2014-10-05 21:35:32,068::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:35:32,163::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:35:32,163::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-72::DEBUG::2014-10-05 21:35:32,164::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:35:35,324::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:35:35,325::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-73::DEBUG::2014-10-05 21:35:35,325::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-19::DEBUG::2014-10-05 21:35:35,811::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-19::DEBUG::2014-10-05 21:35:35,816::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000713933 s, 5.7 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:35:36,780::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-18::DEBUG::2014-10-05 21:35:36,784::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.0006568 s, 6.2 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:35:38,411::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:35:38,412::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-74::DEBUG::2014-10-05 21:35:38,413::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:35:41,809::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:35:41,810::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-75::DEBUG::2014-10-05 21:35:41,812::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:35:44,899::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:35:44,900::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-76::DEBUG::2014-10-05 21:35:44,901::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-19::DEBUG::2014-10-05 21:35:45,818::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-19::DEBUG::2014-10-05 21:35:45,822::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000652098 s, 6.3 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:35:46,786::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-18::DEBUG::2014-10-05 21:35:46,790::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000553368 s, 7.4 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:35:48,014::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:35:48,015::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-77::DEBUG::2014-10-05 21:35:48,017::task::595::Storage.TaskManager.Task::(_updateState) Task=`a01ca1c3-4f2a-46c0-b900-b17281b1ea86`::moving from state init -> state preparing >Thread-77::INFO::2014-10-05 21:35:48,017::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-77::INFO::2014-10-05 21:35:48,017::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000553368', 'lastCheck': '1.2', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000652098', 'lastCheck': '2.2', 'valid': True}} >Thread-77::DEBUG::2014-10-05 21:35:48,017::task::1191::Storage.TaskManager.Task::(prepare) Task=`a01ca1c3-4f2a-46c0-b900-b17281b1ea86`::finished: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000553368', 'lastCheck': '1.2', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000652098', 'lastCheck': '2.2', 'valid': True}} >Thread-77::DEBUG::2014-10-05 21:35:48,017::task::595::Storage.TaskManager.Task::(_updateState) Task=`a01ca1c3-4f2a-46c0-b900-b17281b1ea86`::moving from state preparing -> state finished >Thread-77::DEBUG::2014-10-05 21:35:48,017::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-77::DEBUG::2014-10-05 21:35:48,017::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-77::DEBUG::2014-10-05 21:35:48,017::task::993::Storage.TaskManager.Task::(_decref) Task=`a01ca1c3-4f2a-46c0-b900-b17281b1ea86`::ref 0 aborting False >Thread-77::DEBUG::2014-10-05 21:35:48,019::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:35:48,107::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:35:48,108::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-78::DEBUG::2014-10-05 21:35:48,109::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:35:51,349::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:35:51,351::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-79::DEBUG::2014-10-05 21:35:51,352::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:35:54,444::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:35:54,444::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-80::DEBUG::2014-10-05 21:35:54,445::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-19::DEBUG::2014-10-05 21:35:55,824::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-19::DEBUG::2014-10-05 21:35:55,829::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000693552 s, 5.9 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:35:56,791::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-18::DEBUG::2014-10-05 21:35:56,796::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000598477 s, 6.8 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:35:57,541::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:35:57,543::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-81::DEBUG::2014-10-05 21:35:57,544::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:36:00,636::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:36:00,636::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-82::DEBUG::2014-10-05 21:36:00,637::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:36:03,738::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:36:03,739::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-83::DEBUG::2014-10-05 21:36:03,741::task::595::Storage.TaskManager.Task::(_updateState) Task=`0b6de2c7-c8f0-4ecc-a6fa-18303423eede`::moving from state init -> state preparing >Thread-83::INFO::2014-10-05 21:36:03,741::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-83::INFO::2014-10-05 21:36:03,741::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000598477', 'lastCheck': '6.9', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000693552', 'lastCheck': '7.9', 'valid': True}} >Thread-83::DEBUG::2014-10-05 21:36:03,741::task::1191::Storage.TaskManager.Task::(prepare) Task=`0b6de2c7-c8f0-4ecc-a6fa-18303423eede`::finished: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000598477', 'lastCheck': '6.9', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000693552', 'lastCheck': '7.9', 'valid': True}} >Thread-83::DEBUG::2014-10-05 21:36:03,741::task::595::Storage.TaskManager.Task::(_updateState) Task=`0b6de2c7-c8f0-4ecc-a6fa-18303423eede`::moving from state preparing -> state finished >Thread-83::DEBUG::2014-10-05 21:36:03,741::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-83::DEBUG::2014-10-05 21:36:03,741::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-83::DEBUG::2014-10-05 21:36:03,741::task::993::Storage.TaskManager.Task::(_decref) Task=`0b6de2c7-c8f0-4ecc-a6fa-18303423eede`::ref 0 aborting False >Thread-83::DEBUG::2014-10-05 21:36:03,743::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:36:03,831::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:36:03,832::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-84::DEBUG::2014-10-05 21:36:03,840::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-19::DEBUG::2014-10-05 21:36:05,830::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-19::DEBUG::2014-10-05 21:36:05,835::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000757731 s, 5.4 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:36:06,797::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-18::DEBUG::2014-10-05 21:36:06,801::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000547384 s, 7.5 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:36:06,997::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:36:06,998::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-85::DEBUG::2014-10-05 21:36:06,999::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:36:10,087::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:36:10,088::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-86::DEBUG::2014-10-05 21:36:10,088::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:36:13,175::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:36:13,176::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-87::DEBUG::2014-10-05 21:36:13,177::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-19::DEBUG::2014-10-05 21:36:15,836::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-19::DEBUG::2014-10-05 21:36:15,841::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000734973 s, 5.6 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:36:16,268::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:36:16,269::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-88::DEBUG::2014-10-05 21:36:16,270::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-18::DEBUG::2014-10-05 21:36:16,802::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-18::DEBUG::2014-10-05 21:36:16,806::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000572016 s, 7.2 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:36:19,357::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:36:19,358::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-89::DEBUG::2014-10-05 21:36:19,360::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5c20b67-1de4-48b9-afe3-1bf87cedbce9`::moving from state init -> state preparing >Thread-89::INFO::2014-10-05 21:36:19,360::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-89::INFO::2014-10-05 21:36:19,360::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000572016', 'lastCheck': '2.6', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000734973', 'lastCheck': '3.5', 'valid': True}} >Thread-89::DEBUG::2014-10-05 21:36:19,360::task::1191::Storage.TaskManager.Task::(prepare) Task=`d5c20b67-1de4-48b9-afe3-1bf87cedbce9`::finished: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000572016', 'lastCheck': '2.6', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000734973', 'lastCheck': '3.5', 'valid': True}} >Thread-89::DEBUG::2014-10-05 21:36:19,360::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5c20b67-1de4-48b9-afe3-1bf87cedbce9`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-10-05 21:36:19,360::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-10-05 21:36:19,360::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-10-05 21:36:19,360::task::993::Storage.TaskManager.Task::(_decref) Task=`d5c20b67-1de4-48b9-afe3-1bf87cedbce9`::ref 0 aborting False >Thread-89::DEBUG::2014-10-05 21:36:19,362::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:36:19,467::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:36:19,468::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-90::DEBUG::2014-10-05 21:36:19,468::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:36:22,617::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:36:22,618::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-91::DEBUG::2014-10-05 21:36:22,619::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:36:25,709::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:36:25,710::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-92::DEBUG::2014-10-05 21:36:25,713::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-19::DEBUG::2014-10-05 21:36:25,843::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-19::DEBUG::2014-10-05 21:36:25,847::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000746898 s, 5.5 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:36:26,808::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-18::DEBUG::2014-10-05 21:36:26,812::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000883808 s, 4.6 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:36:28,800::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:36:28,801::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-93::DEBUG::2014-10-05 21:36:28,802::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:36:31,890::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:36:31,891::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-94::DEBUG::2014-10-05 21:36:31,892::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:36:34,979::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:36:34,982::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-95::DEBUG::2014-10-05 21:36:34,983::task::595::Storage.TaskManager.Task::(_updateState) Task=`67d2b3d1-c88c-4379-a790-2d1891dc2a13`::moving from state init -> state preparing >Thread-95::INFO::2014-10-05 21:36:34,984::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-10-05 21:36:34,984::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000883808', 'lastCheck': '8.2', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000746898', 'lastCheck': '9.1', 'valid': True}} >Thread-95::DEBUG::2014-10-05 21:36:34,984::task::1191::Storage.TaskManager.Task::(prepare) Task=`67d2b3d1-c88c-4379-a790-2d1891dc2a13`::finished: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000883808', 'lastCheck': '8.2', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000746898', 'lastCheck': '9.1', 'valid': True}} >Thread-95::DEBUG::2014-10-05 21:36:34,984::task::595::Storage.TaskManager.Task::(_updateState) Task=`67d2b3d1-c88c-4379-a790-2d1891dc2a13`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-10-05 21:36:34,984::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-10-05 21:36:34,984::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-10-05 21:36:34,984::task::993::Storage.TaskManager.Task::(_decref) Task=`67d2b3d1-c88c-4379-a790-2d1891dc2a13`::ref 0 aborting False >Thread-95::DEBUG::2014-10-05 21:36:34,986::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:36:35,078::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:36:35,079::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-96::DEBUG::2014-10-05 21:36:35,080::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-19::DEBUG::2014-10-05 21:36:35,849::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-19::DEBUG::2014-10-05 21:36:35,854::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000782837 s, 5.2 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:36:36,814::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-18::DEBUG::2014-10-05 21:36:36,818::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000590532 s, 6.9 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:36:38,256::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:36:38,257::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-97::DEBUG::2014-10-05 21:36:38,258::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:36:41,345::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:36:41,346::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-98::DEBUG::2014-10-05 21:36:41,347::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:36:44,438::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:36:44,439::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-99::DEBUG::2014-10-05 21:36:44,440::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-19::DEBUG::2014-10-05 21:36:45,856::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-19::DEBUG::2014-10-05 21:36:45,860::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000790754 s, 5.2 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:36:46,819::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-18::DEBUG::2014-10-05 21:36:46,824::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000596486 s, 6.9 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:36:47,534::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:36:47,535::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-100::DEBUG::2014-10-05 21:36:47,537::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:36:50,622::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:36:50,623::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-101::DEBUG::2014-10-05 21:36:50,624::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f6af9e0-b3b8-4d19-b95f-7f37570dfe86`::moving from state init -> state preparing >Thread-101::INFO::2014-10-05 21:36:50,624::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-101::INFO::2014-10-05 21:36:50,624::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000596486', 'lastCheck': '3.8', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000790754', 'lastCheck': '4.8', 'valid': True}} >Thread-101::DEBUG::2014-10-05 21:36:50,624::task::1191::Storage.TaskManager.Task::(prepare) Task=`1f6af9e0-b3b8-4d19-b95f-7f37570dfe86`::finished: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000596486', 'lastCheck': '3.8', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000790754', 'lastCheck': '4.8', 'valid': True}} >Thread-101::DEBUG::2014-10-05 21:36:50,624::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f6af9e0-b3b8-4d19-b95f-7f37570dfe86`::moving from state preparing -> state finished >Thread-101::DEBUG::2014-10-05 21:36:50,625::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-101::DEBUG::2014-10-05 21:36:50,625::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-101::DEBUG::2014-10-05 21:36:50,625::task::993::Storage.TaskManager.Task::(_decref) Task=`1f6af9e0-b3b8-4d19-b95f-7f37570dfe86`::ref 0 aborting False >Thread-101::DEBUG::2014-10-05 21:36:50,626::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:36:50,716::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:36:50,716::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-102::DEBUG::2014-10-05 21:36:50,717::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:36:53,901::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:36:53,903::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-103::DEBUG::2014-10-05 21:36:53,905::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-19::DEBUG::2014-10-05 21:36:55,862::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-19::DEBUG::2014-10-05 21:36:55,867::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000808858 s, 5.1 MB/s\n'; <rc> = 0 >Thread-18::DEBUG::2014-10-05 21:36:56,826::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-18::DEBUG::2014-10-05 21:36:56,830::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000580458 s, 7.1 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:36:56,992::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:36:56,993::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-104::DEBUG::2014-10-05 21:36:56,995::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:37:00,087::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:37:00,088::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-105::DEBUG::2014-10-05 21:37:00,089::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:37:03,183::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:37:03,184::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-106::DEBUG::2014-10-05 21:37:03,185::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-19::DEBUG::2014-10-05 21:37:05,868::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/ff559f46-c495-4f6b-901c-2a624042a050/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-19::DEBUG::2014-10-05 21:37:05,873::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000755155 s, 5.4 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:37:06,272::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:37:06,273::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-107::DEBUG::2014-10-05 21:37:06,274::task::595::Storage.TaskManager.Task::(_updateState) Task=`e35600be-68a8-4566-b5ca-c8aafd1e7732`::moving from state init -> state preparing >Thread-107::INFO::2014-10-05 21:37:06,275::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-107::INFO::2014-10-05 21:37:06,275::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000580458', 'lastCheck': '9.4', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000755155', 'lastCheck': '0.4', 'valid': True}} >Thread-107::DEBUG::2014-10-05 21:37:06,275::task::1191::Storage.TaskManager.Task::(prepare) Task=`e35600be-68a8-4566-b5ca-c8aafd1e7732`::finished: {u'c73a3ced-4b41-4b4e-93f7-494d63e7e7ae': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000580458', 'lastCheck': '9.4', 'valid': True}, u'ff559f46-c495-4f6b-901c-2a624042a050': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000755155', 'lastCheck': '0.4', 'valid': True}} >Thread-107::DEBUG::2014-10-05 21:37:06,275::task::595::Storage.TaskManager.Task::(_updateState) Task=`e35600be-68a8-4566-b5ca-c8aafd1e7732`::moving from state preparing -> state finished >Thread-107::DEBUG::2014-10-05 21:37:06,275::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-107::DEBUG::2014-10-05 21:37:06,275::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-107::DEBUG::2014-10-05 21:37:06,275::task::993::Storage.TaskManager.Task::(_decref) Task=`e35600be-68a8-4566-b5ca-c8aafd1e7732`::ref 0 aborting False >Thread-107::DEBUG::2014-10-05 21:37:06,277::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:37:06,366::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:37:06,366::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-108::DEBUG::2014-10-05 21:37:06,367::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-18::DEBUG::2014-10-05 21:37:06,831::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/c73a3ced-4b41-4b4e-93f7-494d63e7e7ae/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-18::DEBUG::2014-10-05 21:37:06,836::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000555443 s, 7.4 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:37:09,596::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:37:09,598::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-109::DEBUG::2014-10-05 21:37:09,599::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-05 21:37:12,690::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-05 21:37:12,691::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-110::DEBUG::2014-10-05 21:37:12,693::stompReactor::163::yajsonrpc.StompServer::(send) Sending response
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Raw
Actions:
View
Attachments on
bug 1149517
:
944065
| 944066 |
944067
|
944068