Attachment 829271 Details for Bug 1034777 – vdsm log might be also useful

vdsm log might be also useful

vdsm.log (text/x-log), 232.33 KB, created by Lukas Svaty on 2013-11-26 13:42:55 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Lukas Svaty

Created: 2013-11-26 13:42:55 UTC

Size: 232.33 KB

patch

obsolete

>MainThread::DEBUG::2013-11-26 13:46:08,449::vdsm::45::vds::(sigtermHandler) Received signal 15
>MainThread::INFO::2013-11-26 13:46:08,529::vmChannels::183::vds::(stop) VM channels listener was stopped.
>MainThread::INFO::2013-11-26 13:46:08,530::momIF::81::MOM::(stop) Shutting down MOM
>MainThread::DEBUG::2013-11-26 13:46:08,531::task::579::TaskManager.Task::(_updateState) Task=`3dc0881a-31d7-4f09-acdd-16d9b30f05f5`::moving from state init -> state preparing
>MainThread::INFO::2013-11-26 13:46:08,531::logUtils::44::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None)
>Thread-11::DEBUG::2013-11-26 13:46:08,534::storageServer::698::ConnectionMonitor::(_monitorConnections) Monitoring stopped
>VM Channels Listener::INFO::2013-11-26 13:46:08,764::vmChannels::178::vds::(run) VM channels listener thread has ended.
>MainThread::DEBUG::2013-11-26 13:46:09,475::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks
>MainThread::INFO::2013-11-26 13:46:09,476::logUtils::47::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None
>MainThread::DEBUG::2013-11-26 13:46:09,476::task::1168::TaskManager.Task::(prepare) Task=`3dc0881a-31d7-4f09-acdd-16d9b30f05f5`::finished: None
>MainThread::DEBUG::2013-11-26 13:46:09,476::task::579::TaskManager.Task::(_updateState) Task=`3dc0881a-31d7-4f09-acdd-16d9b30f05f5`::moving from state preparing -> state finished
>MainThread::DEBUG::2013-11-26 13:46:09,476::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>MainThread::DEBUG::2013-11-26 13:46:09,477::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>MainThread::DEBUG::2013-11-26 13:46:09,477::task::974::TaskManager.Task::(_decref) Task=`3dc0881a-31d7-4f09-acdd-16d9b30f05f5`::ref 0 aborting False
>MainThread::INFO::2013-11-26 13:46:09,477::vdsm::107::vds::(run) VDSM main thread ended. Waiting for 15 other threads...
>MainThread::INFO::2013-11-26 13:46:09,477::vdsm::111::vds::(run) <_MainThread(MainThread, started 140671215216384)>
>MainThread::INFO::2013-11-26 13:46:09,477::vdsm::111::vds::(run) <Thread(libvirtEventLoop, started daemon 140671125325568)>
>MainThread::INFO::2013-11-26 13:46:09,478::vdsm::111::vds::(run) <WorkerThread(Thread-2, started daemon 140670983468800)>
>MainThread::INFO::2013-11-26 13:46:09,478::vdsm::111::vds::(run) <PolicyEngine(PolicyEngine, started daemon 140669989373696)>
>MainThread::INFO::2013-11-26 13:46:09,478::vdsm::111::vds::(run) <GuestManager(GuestManager, started daemon 140669999863552)>
>MainThread::INFO::2013-11-26 13:46:09,478::vdsm::111::vds::(run) <WorkerThread(Thread-8, started daemon 140670513706752)>
>MainThread::INFO::2013-11-26 13:46:09,478::vdsm::111::vds::(run) <WorkerThread(Thread-7, started daemon 140670931019520)>
>MainThread::INFO::2013-11-26 13:46:09,479::vdsm::111::vds::(run) <WorkerThread(Thread-6, started daemon 140670941509376)>
>MainThread::INFO::2013-11-26 13:46:09,479::vdsm::111::vds::(run) <WorkerThread(Thread-3, started daemon 140670972978944)>
>MainThread::INFO::2013-11-26 13:46:09,479::vdsm::111::vds::(run) <WorkerThread(Thread-10, started daemon 140670492727040)>
>MainThread::INFO::2013-11-26 13:46:09,479::vdsm::111::vds::(run) <WorkerThread(Thread-9, started daemon 140670503216896)>
>MainThread::INFO::2013-11-26 13:46:09,479::momIF::81::MOM::(stop) Shutting down MOM
>MainThread::INFO::2013-11-26 13:46:09,479::vdsm::111::vds::(run) <MomThread(MOM, started 140670043944704)>
>MainThread::INFO::2013-11-26 13:46:09,480::vdsm::111::vds::(run) <WorkerThread(Thread-5, started daemon 140670951999232)>
>MainThread::INFO::2013-11-26 13:46:09,480::vdsm::111::vds::(run) <WorkerThread(Thread-1, started daemon 140671067784960)>
>MainThread::INFO::2013-11-26 13:46:09,480::vdsm::111::vds::(run) <HostMonitor(HostMonitor, started daemon 140670033454848)>
>MainThread::INFO::2013-11-26 13:46:09,480::vdsm::111::vds::(run) <WorkerThread(Thread-4, started daemon 140670962489088)>
>MainThread::INFO::2013-11-26 13:46:13,427::vdsm::101::vds::(run) (PID: 6828) I am the actual vdsm 4.13.0-0.9.beta1.el6ev slot-5.rhev.lab.eng.brq.redhat.com (2.6.32-431.el6.x86_64)
>MainThread::DEBUG::2013-11-26 13:46:13,776::resourceManager::420::ResourceManager::(registerNamespace) Registering namespace 'Storage'
>MainThread::DEBUG::2013-11-26 13:46:13,776::threadPool::35::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0
>MainThread::WARNING::2013-11-26 13:46:13,781::fileUtils::167::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt already exists
>MainThread::DEBUG::2013-11-26 13:46:14,638::supervdsm::77::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm
>MainThread::DEBUG::2013-11-26 13:46:14,659::multipath::144::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving
>MainThread::DEBUG::2013-11-26 13:46:14,659::hsm::426::Storage.Misc.excCmd::(__validateLvmLockingType) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None)
>MainThread::DEBUG::2013-11-26 13:46:14,827::hsm::426::Storage.Misc.excCmd::(__validateLvmLockingType) SUCCESS: <err> = ''; <rc> = 0
>MainThread::DEBUG::2013-11-26 13:46:14,828::hsm::450::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center'
>MainThread::DEBUG::2013-11-26 13:46:14,831::hsm::482::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt']
>MainThread::DEBUG::2013-11-26 13:46:14,831::hsm::483::Storage.HSM::(__cleanStorageRepository) Mount list: ['/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs02', '/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_nfs01']
>MainThread::DEBUG::2013-11-26 13:46:14,831::hsm::485::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers
>MainThread::DEBUG::2013-11-26 13:46:14,832::hsm::528::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center'
>MainThread::INFO::2013-11-26 13:46:14,833::dispatcher::98::Storage.Dispatcher::(__init__) Starting StorageDispatcher...
>storageRefresh::DEBUG::2013-11-26 13:46:14,834::lvm::338::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex
>MainThread::DEBUG::2013-11-26 13:46:14,834::task::579::TaskManager.Task::(_updateState) Task=`385014d9-9a94-47a3-a2ba-27e74ab068d8`::moving from state init -> state preparing
>MainThread::INFO::2013-11-26 13:46:14,835::logUtils::44::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback(callbackFunc=<bound method clientIF.contEIOVms of <clientIF.clientIF instance at 0x7fb0f0c983f8>>)
>MainThread::INFO::2013-11-26 13:46:14,835::logUtils::47::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback, Return response: None
>MainThread::DEBUG::2013-11-26 13:46:14,835::task::1168::TaskManager.Task::(prepare) Task=`385014d9-9a94-47a3-a2ba-27e74ab068d8`::finished: None
>MainThread::DEBUG::2013-11-26 13:46:14,836::task::579::TaskManager.Task::(_updateState) Task=`385014d9-9a94-47a3-a2ba-27e74ab068d8`::moving from state preparing -> state finished
>MainThread::DEBUG::2013-11-26 13:46:14,836::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>MainThread::DEBUG::2013-11-26 13:46:14,836::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>MainThread::DEBUG::2013-11-26 13:46:14,836::task::974::TaskManager.Task::(_decref) Task=`385014d9-9a94-47a3-a2ba-27e74ab068d8`::ref 0 aborting False
>storageRefresh::DEBUG::2013-11-26 13:46:14,838::lvm::309::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | --ignoreskippedcluster -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None)
>MainThread::INFO::2013-11-26 13:46:14,846::momIF::47::MOM::(__init__) Starting up MOM
>clientIFinit::DEBUG::2013-11-26 13:46:14,848::libvirtconnection::131::libvirtconnection::(get) trying to connect libvirt
>MainThread::INFO::2013-11-26 13:46:14,848::vmChannels::187::vds::(settimeout) Setting channels' timeout to 30 seconds.
>VM Channels Listener::INFO::2013-11-26 13:46:14,857::vmChannels::170::vds::(run) Starting VM channels listener thread.
>storageRefresh::DEBUG::2013-11-26 13:46:14,999::lvm::309::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>storageRefresh::DEBUG::2013-11-26 13:46:15,001::lvm::363::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex
>storageRefresh::DEBUG::2013-11-26 13:46:15,002::lvm::386::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>storageRefresh::DEBUG::2013-11-26 13:46:15,005::lvm::309::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None)
>storageRefresh::DEBUG::2013-11-26 13:46:15,222::lvm::309::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = '  No volume groups found\n'; <rc> = 0
>storageRefresh::DEBUG::2013-11-26 13:46:15,223::lvm::428::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>storageRefresh::DEBUG::2013-11-26 13:46:15,224::lvm::309::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | --ignoreskippedcluster -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None)
>storageRefresh::DEBUG::2013-11-26 13:46:15,436::lvm::309::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = '  No volume groups found\n'; <rc> = 0
>storageRefresh::DEBUG::2013-11-26 13:46:15,437::misc::809::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage)
>storageRefresh::DEBUG::2013-11-26 13:46:15,438::misc::811::SamplingMethod::(__call__) Got in to sampling method
>storageRefresh::DEBUG::2013-11-26 13:46:15,438::misc::809::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan)
>storageRefresh::DEBUG::2013-11-26 13:46:15,438::misc::811::SamplingMethod::(__call__) Got in to sampling method
>storageRefresh::DEBUG::2013-11-26 13:46:15,439::iscsiadm::91::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None)
>storageRefresh::DEBUG::2013-11-26 13:46:15,454::iscsiadm::91::Storage.Misc.excCmd::(_runCmd) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21
>storageRefresh::DEBUG::2013-11-26 13:46:15,455::misc::819::SamplingMethod::(__call__) Returning last result
>Thread-13::DEBUG::2013-11-26 13:46:15,461::BindingXMLRPC::984::vds::(wrapper) client [127.0.0.1]::call getHardwareInfo with () {}
>Thread-13::DEBUG::2013-11-26 13:46:15,465::BindingXMLRPC::991::vds::(wrapper) return getHardwareInfo with {'status': {'message': 'Done', 'code': 0}, 'info': {'systemProductName': 'PowerEdge M610', 'systemSerialNumber': '4RLXC5J', 'systemFamily': 'Not Specified', 'systemVersion': 'Not Specified', 'systemUUID': '4C4C4544-0052-4C10-8058-B4C04F43354A', 'systemManufacturer': 'Dell Inc.'}}
>Thread-14::DEBUG::2013-11-26 13:46:15,522::BindingXMLRPC::177::vds::(wrapper) client [127.0.0.1]
>Thread-14::DEBUG::2013-11-26 13:46:15,523::task::579::TaskManager.Task::(_updateState) Task=`79f8acce-3183-4d10-a54e-b1b06639f9d9`::moving from state init -> state preparing
>Thread-14::INFO::2013-11-26 13:46:15,523::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None)
>Thread-14::INFO::2013-11-26 13:46:15,524::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []}
>Thread-14::DEBUG::2013-11-26 13:46:15,524::task::1168::TaskManager.Task::(prepare) Task=`79f8acce-3183-4d10-a54e-b1b06639f9d9`::finished: {'poollist': []}
>Thread-14::DEBUG::2013-11-26 13:46:15,524::task::579::TaskManager.Task::(_updateState) Task=`79f8acce-3183-4d10-a54e-b1b06639f9d9`::moving from state preparing -> state finished
>Thread-14::DEBUG::2013-11-26 13:46:15,525::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-14::DEBUG::2013-11-26 13:46:15,525::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-14::DEBUG::2013-11-26 13:46:15,525::task::974::TaskManager.Task::(_decref) Task=`79f8acce-3183-4d10-a54e-b1b06639f9d9`::ref 0 aborting False
>Thread-15::DEBUG::2013-11-26 13:46:15,581::BindingXMLRPC::177::vds::(wrapper) client [127.0.0.1]
>Thread-15::DEBUG::2013-11-26 13:46:15,582::task::579::TaskManager.Task::(_updateState) Task=`9be3cb48-e2e9-4be5-b238-7e7500ed5ef5`::moving from state init -> state preparing
>Thread-15::INFO::2013-11-26 13:46:15,582::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine', 'iqn': '', 'portal': '', 'user': 'kvm', 'password': '******', 'id': '2870d98f-1a5f-4ff4-8ed4-886a6185f785', 'port': ''}], options=None)
>Thread-15::DEBUG::2013-11-26 13:46:15,587::mount::226::Storage.Misc.excCmd::(_runcmd) '/usr/bin/sudo -n /bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6,nfsvers=3 10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine' (cwd None)
>Thread-15::DEBUG::2013-11-26 13:46:15,832::hsm::2313::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine
>Thread-15::DEBUG::2013-11-26 13:46:15,834::hsm::2337::Storage.HSM::(__prefetchDomains) Found SD uuids: ()
>Thread-15::DEBUG::2013-11-26 13:46:15,834::hsm::2393::Storage.HSM::(connectStorageServer) knownSDs: {}
>Thread-15::INFO::2013-11-26 13:46:15,834::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '2870d98f-1a5f-4ff4-8ed4-886a6185f785'}]}
>Thread-15::DEBUG::2013-11-26 13:46:15,835::task::1168::TaskManager.Task::(prepare) Task=`9be3cb48-e2e9-4be5-b238-7e7500ed5ef5`::finished: {'statuslist': [{'status': 0, 'id': '2870d98f-1a5f-4ff4-8ed4-886a6185f785'}]}
>Thread-15::DEBUG::2013-11-26 13:46:15,835::task::579::TaskManager.Task::(_updateState) Task=`9be3cb48-e2e9-4be5-b238-7e7500ed5ef5`::moving from state preparing -> state finished
>Thread-15::DEBUG::2013-11-26 13:46:15,835::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15::DEBUG::2013-11-26 13:46:15,836::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15::DEBUG::2013-11-26 13:46:15,836::task::974::TaskManager.Task::(_decref) Task=`9be3cb48-e2e9-4be5-b238-7e7500ed5ef5`::ref 0 aborting False
>Thread-16::DEBUG::2013-11-26 13:46:15,896::BindingXMLRPC::177::vds::(wrapper) client [127.0.0.1]
>Thread-16::DEBUG::2013-11-26 13:46:15,896::task::579::TaskManager.Task::(_updateState) Task=`a2d5716c-9626-4430-a319-25b9c36720ea`::moving from state init -> state preparing
>Thread-16::INFO::2013-11-26 13:46:15,897::logUtils::44::dispatcher::(wrapper) Run and protect: createStorageDomain(storageType=1, sdUUID='23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', domainName='hosted_storage', typeSpecificArg='10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine', domClass=1, domVersion=3, options=None)
>Thread-16::DEBUG::2013-11-26 13:46:15,897::misc::809::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage)
>storageRefresh::DEBUG::2013-11-26 13:46:18,475::multipath::112::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None)
>storageRefresh::DEBUG::2013-11-26 13:46:18,678::multipath::112::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0
>storageRefresh::DEBUG::2013-11-26 13:46:18,679::lvm::510::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex
>storageRefresh::DEBUG::2013-11-26 13:46:18,679::lvm::512::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex
>storageRefresh::DEBUG::2013-11-26 13:46:18,679::lvm::521::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex
>storageRefresh::DEBUG::2013-11-26 13:46:18,680::lvm::523::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex
>storageRefresh::DEBUG::2013-11-26 13:46:18,680::lvm::541::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex
>storageRefresh::DEBUG::2013-11-26 13:46:18,680::lvm::543::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex
>storageRefresh::DEBUG::2013-11-26 13:46:18,681::misc::819::SamplingMethod::(__call__) Returning last result
>Thread-16::DEBUG::2013-11-26 13:46:18,681::misc::811::SamplingMethod::(__call__) Got in to sampling method
>storageRefresh::WARNING::2013-11-26 13:46:18,682::fileUtils::167::Storage.fileUtils::(createdir) Dir /rhev/data-center/hsm-tasks already exists
>Thread-16::DEBUG::2013-11-26 13:46:18,683::misc::809::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan)
>Thread-16::DEBUG::2013-11-26 13:46:18,684::misc::811::SamplingMethod::(__call__) Got in to sampling method
>Thread-16::DEBUG::2013-11-26 13:46:18,685::iscsiadm::91::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None)
>Thread-16::DEBUG::2013-11-26 13:46:18,701::iscsiadm::91::Storage.Misc.excCmd::(_runCmd) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21
>Thread-16::DEBUG::2013-11-26 13:46:18,705::misc::819::SamplingMethod::(__call__) Returning last result
>Thread-16::DEBUG::2013-11-26 13:46:22,745::multipath::112::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None)
>Thread-16::DEBUG::2013-11-26 13:46:22,875::multipath::112::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0
>Thread-16::DEBUG::2013-11-26 13:46:22,876::lvm::510::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-16::DEBUG::2013-11-26 13:46:22,876::lvm::512::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-16::DEBUG::2013-11-26 13:46:22,877::lvm::521::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-16::DEBUG::2013-11-26 13:46:22,877::lvm::523::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-16::DEBUG::2013-11-26 13:46:22,878::lvm::541::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-16::DEBUG::2013-11-26 13:46:22,878::lvm::543::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-16::DEBUG::2013-11-26 13:46:22,879::misc::819::SamplingMethod::(__call__) Returning last result
>Thread-16::ERROR::2013-11-26 13:46:22,879::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-16::ERROR::2013-11-26 13:46:22,879::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-16::DEBUG::2013-11-26 13:46:22,880::lvm::386::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-16::DEBUG::2013-11-26 13:46:22,882::lvm::309::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c' (cwd None)
>Thread-16::DEBUG::2013-11-26 13:46:23,130::lvm::309::Storage.Misc.excCmd::(cmd) FAILED: <err> = '  Volume group "23b22ad4-6d4c-4cdc-b25e-6e9efef7004c" not found\n  Skipping volume group 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c\n'; <rc> = 5
>Thread-16::WARNING::2013-11-26 13:46:23,131::lvm::391::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "23b22ad4-6d4c-4cdc-b25e-6e9efef7004c" not found', '  Skipping volume group 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c']
>Thread-16::DEBUG::2013-11-26 13:46:23,132::lvm::428::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-16::ERROR::2013-11-26 13:46:23,270::sdc::143::Storage.StorageDomainCache::(_findDomain) domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c not found
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain
>    dom = findMethod(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 171, in _findUnfetchedDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: ('23b22ad4-6d4c-4cdc-b25e-6e9efef7004c',)
>Thread-16::INFO::2013-11-26 13:46:23,271::nfsSD::69::Storage.StorageDomain::(create) sdUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c domainName=hosted_storage remotePath=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine domClass=1
>Thread-16::DEBUG::2013-11-26 13:46:23,715::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend
>Thread-16::DEBUG::2013-11-26 13:46:23,723::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=[]
>Thread-16::WARNING::2013-11-26 13:46:23,723::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>Thread-16::DEBUG::2013-11-26 13:46:23,724::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction
>Thread-16::DEBUG::2013-11-26 13:46:23,724::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes
>Thread-16::DEBUG::2013-11-26 13:46:23,724::persistentDict::299::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine', 'ROLE=Regular', 'SDUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=472a650f5ae89ba476db41e8ca7f20425712d937']
>Thread-16::DEBUG::2013-11-26 13:46:23,788::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction
>Thread-16::DEBUG::2013-11-26 13:46:23,790::fileSD::154::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-16::DEBUG::2013-11-26 13:46:23,791::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend
>Thread-16::DEBUG::2013-11-26 13:46:23,802::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine', 'ROLE=Regular', 'SDUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=472a650f5ae89ba476db41e8ca7f20425712d937']
>Thread-16::DEBUG::2013-11-26 13:46:23,807::fileSD::572::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images []
>Thread-16::DEBUG::2013-11-26 13:46:23,808::resourceManager::420::ResourceManager::(registerNamespace) Registering namespace '23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_imageNS'
>Thread-16::DEBUG::2013-11-26 13:46:23,808::resourceManager::420::ResourceManager::(registerNamespace) Registering namespace '23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_volumeNS'
>Thread-16::DEBUG::2013-11-26 13:46:23,809::clusterlock::137::initSANLock::(initSANLock) Initializing SANLock for domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-16::DEBUG::2013-11-26 13:46:23,938::sd::424::Storage.StorageDomain::(initSPMlease) lease initialized successfully
>Thread-16::DEBUG::2013-11-26 13:46:23,938::hsm::2632::Storage.HSM::(createStorageDomain) knownSDs: {23b22ad4-6d4c-4cdc-b25e-6e9efef7004c: storage.nfsSD.findDomain}
>Thread-16::INFO::2013-11-26 13:46:23,938::logUtils::47::dispatcher::(wrapper) Run and protect: createStorageDomain, Return response: None
>Thread-16::DEBUG::2013-11-26 13:46:23,938::task::1168::TaskManager.Task::(prepare) Task=`a2d5716c-9626-4430-a319-25b9c36720ea`::finished: None
>Thread-16::DEBUG::2013-11-26 13:46:23,939::task::579::TaskManager.Task::(_updateState) Task=`a2d5716c-9626-4430-a319-25b9c36720ea`::moving from state preparing -> state finished
>Thread-16::DEBUG::2013-11-26 13:46:23,939::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-16::DEBUG::2013-11-26 13:46:23,939::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-16::DEBUG::2013-11-26 13:46:23,939::task::974::TaskManager.Task::(_decref) Task=`a2d5716c-9626-4430-a319-25b9c36720ea`::ref 0 aborting False
>Thread-20::DEBUG::2013-11-26 13:46:23,999::BindingXMLRPC::177::vds::(wrapper) client [127.0.0.1]
>Thread-20::DEBUG::2013-11-26 13:46:24,000::task::579::TaskManager.Task::(_updateState) Task=`c2798997-708e-4d84-90c7-738b8c54b7c0`::moving from state init -> state preparing
>Thread-20::INFO::2013-11-26 13:46:24,000::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-20::INFO::2013-11-26 13:46:24,001::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {}
>Thread-20::DEBUG::2013-11-26 13:46:24,001::task::1168::TaskManager.Task::(prepare) Task=`c2798997-708e-4d84-90c7-738b8c54b7c0`::finished: {}
>Thread-20::DEBUG::2013-11-26 13:46:24,001::task::579::TaskManager.Task::(_updateState) Task=`c2798997-708e-4d84-90c7-738b8c54b7c0`::moving from state preparing -> state finished
>Thread-20::DEBUG::2013-11-26 13:46:24,001::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-20::DEBUG::2013-11-26 13:46:24,001::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-20::DEBUG::2013-11-26 13:46:24,002::task::974::TaskManager.Task::(_decref) Task=`c2798997-708e-4d84-90c7-738b8c54b7c0`::ref 0 aborting False
>Thread-21::DEBUG::2013-11-26 13:46:24,056::BindingXMLRPC::177::vds::(wrapper) client [127.0.0.1]
>Thread-21::DEBUG::2013-11-26 13:46:24,057::task::579::TaskManager.Task::(_updateState) Task=`4242d9cd-1887-4832-99b9-8278783f17d9`::moving from state init -> state preparing
>Thread-21::INFO::2013-11-26 13:46:24,057::logUtils::44::dispatcher::(wrapper) Run and protect: getStorageDomainStats(sdUUID='23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', options=None)
>Thread-21::DEBUG::2013-11-26 13:46:24,058::resourceManager::197::ResourceManager.Request::(__init__) ResName=`Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c`ReqID=`e68b57f1-57d7-4c4c-a6a0-28bb4eec031b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2795' at 'getStorageDomainStats'
>Thread-21::DEBUG::2013-11-26 13:46:24,058::resourceManager::541::ResourceManager::(registerResource) Trying to register resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c' for lock type 'shared'
>Thread-21::DEBUG::2013-11-26 13:46:24,058::resourceManager::600::ResourceManager::(registerResource) Resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c' is free. Now locking as 'shared' (1 active user)
>Thread-21::DEBUG::2013-11-26 13:46:24,058::resourceManager::237::ResourceManager.Request::(grant) ResName=`Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c`ReqID=`e68b57f1-57d7-4c4c-a6a0-28bb4eec031b`::Granted request
>Thread-21::DEBUG::2013-11-26 13:46:24,059::task::811::TaskManager.Task::(resourceAcquired) Task=`4242d9cd-1887-4832-99b9-8278783f17d9`::_resourcesAcquired: Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c (shared)
>Thread-21::DEBUG::2013-11-26 13:46:24,059::task::974::TaskManager.Task::(_decref) Task=`4242d9cd-1887-4832-99b9-8278783f17d9`::ref 1 aborting False
>Thread-21::INFO::2013-11-26 13:46:24,061::logUtils::47::dispatcher::(wrapper) Run and protect: getStorageDomainStats, Return response: {'stats': {'mdasize': 0, 'mdathreshold': True, 'mdavalid': True, 'diskfree': '301288390656', 'disktotal': '422758580224', 'mdafree': 0}}
>Thread-21::DEBUG::2013-11-26 13:46:24,061::task::1168::TaskManager.Task::(prepare) Task=`4242d9cd-1887-4832-99b9-8278783f17d9`::finished: {'stats': {'mdasize': 0, 'mdathreshold': True, 'mdavalid': True, 'diskfree': '301288390656', 'disktotal': '422758580224', 'mdafree': 0}}
>Thread-21::DEBUG::2013-11-26 13:46:24,061::task::579::TaskManager.Task::(_updateState) Task=`4242d9cd-1887-4832-99b9-8278783f17d9`::moving from state preparing -> state finished
>Thread-21::DEBUG::2013-11-26 13:46:24,061::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c': < ResourceRef 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', isValid: 'True' obj: 'None'>}
>Thread-21::DEBUG::2013-11-26 13:46:24,062::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-21::DEBUG::2013-11-26 13:46:24,062::resourceManager::615::ResourceManager::(releaseResource) Trying to release resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c'
>Thread-21::DEBUG::2013-11-26 13:46:24,062::resourceManager::634::ResourceManager::(releaseResource) Released resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c' (0 active users)
>Thread-21::DEBUG::2013-11-26 13:46:24,062::resourceManager::640::ResourceManager::(releaseResource) Resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c' is free, finding out if anyone is waiting for it.
>Thread-21::DEBUG::2013-11-26 13:46:24,063::resourceManager::648::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', Clearing records.
>Thread-21::DEBUG::2013-11-26 13:46:24,063::task::974::TaskManager.Task::(_decref) Task=`4242d9cd-1887-4832-99b9-8278783f17d9`::ref 0 aborting False
>Thread-22::DEBUG::2013-11-26 13:46:24,119::BindingXMLRPC::177::vds::(wrapper) client [127.0.0.1]
>Thread-22::DEBUG::2013-11-26 13:46:24,119::task::579::TaskManager.Task::(_updateState) Task=`755ed18c-8b4f-448a-abb8-6e8a7be7d958`::moving from state init -> state preparing
>Thread-22::INFO::2013-11-26 13:46:24,119::logUtils::44::dispatcher::(wrapper) Run and protect: createStoragePool(poolType=None, spUUID='4f04507e-5611-4853-91ff-4384cabd8f34', poolName='hosted_datacenter', masterDom='23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', domList=['23b22ad4-6d4c-4cdc-b25e-6e9efef7004c'], masterVersion=1, lockPolicy=None, lockRenewalIntervalSec=None, leaseTimeSec=None, ioOpTimeoutSec=None, leaseRetries=None, options=None)
>Thread-22::INFO::2013-11-26 13:46:24,120::fileSD::316::Storage.StorageDomain::(validate) sdUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-22::DEBUG::2013-11-26 13:46:24,129::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine', 'ROLE=Regular', 'SDUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=472a650f5ae89ba476db41e8ca7f20425712d937']
>Thread-22::DEBUG::2013-11-26 13:46:24,130::resourceManager::197::ResourceManager.Request::(__init__) ResName=`Storage.4f04507e-5611-4853-91ff-4384cabd8f34`ReqID=`f4e09e2b-292a-4176-b3ae-1f42f6819e80`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '976' at 'createStoragePool'
>Thread-22::DEBUG::2013-11-26 13:46:24,130::resourceManager::541::ResourceManager::(registerResource) Trying to register resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34' for lock type 'exclusive'
>Thread-22::DEBUG::2013-11-26 13:46:24,131::resourceManager::600::ResourceManager::(registerResource) Resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34' is free. Now locking as 'exclusive' (1 active user)
>Thread-22::DEBUG::2013-11-26 13:46:24,131::resourceManager::237::ResourceManager.Request::(grant) ResName=`Storage.4f04507e-5611-4853-91ff-4384cabd8f34`ReqID=`f4e09e2b-292a-4176-b3ae-1f42f6819e80`::Granted request
>Thread-22::DEBUG::2013-11-26 13:46:24,131::task::811::TaskManager.Task::(resourceAcquired) Task=`755ed18c-8b4f-448a-abb8-6e8a7be7d958`::_resourcesAcquired: Storage.4f04507e-5611-4853-91ff-4384cabd8f34 (exclusive)
>Thread-22::DEBUG::2013-11-26 13:46:24,132::task::974::TaskManager.Task::(_decref) Task=`755ed18c-8b4f-448a-abb8-6e8a7be7d958`::ref 1 aborting False
>Thread-22::DEBUG::2013-11-26 13:46:24,132::resourceManager::197::ResourceManager.Request::(__init__) ResName=`Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c`ReqID=`8898320a-2bf8-4807-a56a-d13954568450`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '978' at 'createStoragePool'
>Thread-22::DEBUG::2013-11-26 13:46:24,132::resourceManager::541::ResourceManager::(registerResource) Trying to register resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c' for lock type 'exclusive'
>Thread-22::DEBUG::2013-11-26 13:46:24,133::resourceManager::600::ResourceManager::(registerResource) Resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c' is free. Now locking as 'exclusive' (1 active user)
>Thread-22::DEBUG::2013-11-26 13:46:24,133::resourceManager::237::ResourceManager.Request::(grant) ResName=`Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c`ReqID=`8898320a-2bf8-4807-a56a-d13954568450`::Granted request
>Thread-22::DEBUG::2013-11-26 13:46:24,133::task::811::TaskManager.Task::(resourceAcquired) Task=`755ed18c-8b4f-448a-abb8-6e8a7be7d958`::_resourcesAcquired: Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c (exclusive)
>Thread-22::DEBUG::2013-11-26 13:46:24,133::task::974::TaskManager.Task::(_decref) Task=`755ed18c-8b4f-448a-abb8-6e8a7be7d958`::ref 1 aborting False
>Thread-22::INFO::2013-11-26 13:46:24,134::sp::590::Storage.StoragePool::(create) spUUID=4f04507e-5611-4853-91ff-4384cabd8f34 poolName=hosted_datacenter master_sd=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c domList=['23b22ad4-6d4c-4cdc-b25e-6e9efef7004c'] masterVersion=1 {'LEASETIMESEC': 60, 'LOCKPOLICY': 'ON', 'IOOPTIMEOUTSEC': 10, 'LEASERETRIES': 3, 'LOCKRENEWALINTERVALSEC': 5}
>Thread-22::INFO::2013-11-26 13:46:24,134::fileSD::316::Storage.StorageDomain::(validate) sdUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-22::DEBUG::2013-11-26 13:46:24,143::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine', 'ROLE=Regular', 'SDUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=472a650f5ae89ba476db41e8ca7f20425712d937']
>Thread-22::DEBUG::2013-11-26 13:46:24,153::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine', 'ROLE=Regular', 'SDUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=472a650f5ae89ba476db41e8ca7f20425712d937']
>Thread-22::DEBUG::2013-11-26 13:46:24,154::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction
>Thread-22::DEBUG::2013-11-26 13:46:24,154::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes
>Thread-22::DEBUG::2013-11-26 13:46:24,154::persistentDict::299::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine', 'ROLE=Regular', 'SDUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=8008bb5ee579d704c7d4d81208fec39656fde063']
>Thread-22::DEBUG::2013-11-26 13:46:24,206::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction
>Thread-22::INFO::2013-11-26 13:46:24,206::clusterlock::174::SANLock::(acquireHostId) Acquiring host id for domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c (id: 250)
>Thread-22::DEBUG::2013-11-26 13:46:45,211::clusterlock::192::SANLock::(acquireHostId) Host id for domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c successfully acquired (id: 250)
>Thread-22::INFO::2013-11-26 13:46:45,221::clusterlock::225::SANLock::(acquire) Acquiring cluster lock for domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c (id: 250)
>Thread-22::DEBUG::2013-11-26 13:46:45,289::clusterlock::251::SANLock::(acquire) Cluster lock for domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c successfully acquired (id: 250)
>Thread-22::INFO::2013-11-26 13:46:45,290::sp::758::Storage.StoragePool::(createMaster) setting master domain for spUUID 4f04507e-5611-4853-91ff-4384cabd8f34 on sdUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-22::DEBUG::2013-11-26 13:46:45,290::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction
>Thread-22::DEBUG::2013-11-26 13:46:45,290::persistentDict::160::Storage.PersistentDict::(transaction) Reusing active transaction
>Thread-22::DEBUG::2013-11-26 13:46:45,302::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine', 'ROLE=Regular', 'SDUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=8008bb5ee579d704c7d4d81208fec39656fde063']
>Thread-22::DEBUG::2013-11-26 13:46:45,303::persistentDict::160::Storage.PersistentDict::(transaction) Reusing active transaction
>Thread-22::DEBUG::2013-11-26 13:46:45,304::persistentDict::160::Storage.PersistentDict::(transaction) Reusing active transaction
>Thread-22::DEBUG::2013-11-26 13:46:45,304::persistentDict::160::Storage.PersistentDict::(transaction) Reusing active transaction
>Thread-22::DEBUG::2013-11-26 13:46:45,304::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes
>Thread-22::DEBUG::2013-11-26 13:46:45,305::persistentDict::299::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=4f04507e-5611-4853-91ff-4384cabd8f34', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine', 'ROLE=Master', 'SDUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=551213bd049cc5a04909832cf1427f157b080822']
>Thread-22::DEBUG::2013-11-26 13:46:45,343::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction
>Thread-22::DEBUG::2013-11-26 13:46:45,343::sp::1533::Storage.StoragePool::(getMasterDomain) Master domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c verified, version 1
>Thread-22::DEBUG::2013-11-26 13:46:45,343::misc::809::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads)
>Thread-22::DEBUG::2013-11-26 13:46:45,344::misc::811::SamplingMethod::(__call__) Got in to sampling method
>Thread-22::DEBUG::2013-11-26 13:46:45,354::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=4f04507e-5611-4853-91ff-4384cabd8f34', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine', 'ROLE=Master', 'SDUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=551213bd049cc5a04909832cf1427f157b080822']
>Thread-23::DEBUG::2013-11-26 13:46:45,355::domainMonitor::159::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-22::DEBUG::2013-11-26 13:46:45,355::sp::1566::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `4f04507e-5611-4853-91ff-4384cabd8f34` started monitoring domain `23b22ad4-6d4c-4cdc-b25e-6e9efef7004c`
>Thread-22::DEBUG::2013-11-26 13:46:45,356::misc::819::SamplingMethod::(__call__) Returning last result
>Thread-22::WARNING::2013-11-26 13:46:45,358::fileUtils::167::Storage.fileUtils::(createdir) Dir /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34 already exists
>Thread-22::DEBUG::2013-11-26 13:46:45,367::lvm::386::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-22::DEBUG::2013-11-26 13:46:45,377::lvm::309::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None)
>Thread-23::DEBUG::2013-11-26 13:46:45,391::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-23::DEBUG::2013-11-26 13:46:45,399::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n507 bytes (507 B) copied, 0.000287375 s, 1.8 MB/s\n'; <rc> = 0
>Thread-22::DEBUG::2013-11-26 13:46:45,603::lvm::309::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = '  No volume groups found\n'; <rc> = 0
>Thread-22::DEBUG::2013-11-26 13:46:45,604::lvm::428::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-22::INFO::2013-11-26 13:46:45,613::sp::1196::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c to /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-22::INFO::2013-11-26 13:46:45,614::sp::1196::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c to /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/mastersd
>Thread-22::INFO::2013-11-26 13:46:45,614::clusterlock::255::SANLock::(release) Releasing cluster lock for domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-22::DEBUG::2013-11-26 13:46:45,631::clusterlock::266::SANLock::(release) Cluster lock for domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c successfully released
>Thread-22::INFO::2013-11-26 13:46:45,631::clusterlock::197::SANLock::(releaseHostId) Releasing host id for domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c (id: 250)
>Thread-22::DEBUG::2013-11-26 13:46:48,633::clusterlock::207::SANLock::(releaseHostId) Host id for domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c released successfully (id: 250)
>Thread-23::DEBUG::2013-11-26 13:46:48,633::domainMonitor::169::Storage.DomainMonitorThread::(_monitorLoop) Stopping domain monitor for 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-23::INFO::2013-11-26 13:46:48,633::clusterlock::197::SANLock::(releaseHostId) Releasing host id for domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c (id: 250)
>Thread-23::DEBUG::2013-11-26 13:46:48,634::clusterlock::207::SANLock::(releaseHostId) Host id for domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c released successfully (id: 250)
>Thread-22::INFO::2013-11-26 13:46:48,634::logUtils::47::dispatcher::(wrapper) Run and protect: createStoragePool, Return response: True
>Thread-22::DEBUG::2013-11-26 13:46:48,635::task::1168::TaskManager.Task::(prepare) Task=`755ed18c-8b4f-448a-abb8-6e8a7be7d958`::finished: True
>Thread-22::DEBUG::2013-11-26 13:46:48,635::task::579::TaskManager.Task::(_updateState) Task=`755ed18c-8b4f-448a-abb8-6e8a7be7d958`::moving from state preparing -> state finished
>Thread-22::DEBUG::2013-11-26 13:46:48,635::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c': < ResourceRef 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', isValid: 'True' obj: 'None'>, 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34': < ResourceRef 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34', isValid: 'True' obj: 'None'>}
>Thread-22::DEBUG::2013-11-26 13:46:48,635::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-22::DEBUG::2013-11-26 13:46:48,636::resourceManager::615::ResourceManager::(releaseResource) Trying to release resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c'
>Thread-22::DEBUG::2013-11-26 13:46:48,636::resourceManager::634::ResourceManager::(releaseResource) Released resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c' (0 active users)
>Thread-22::DEBUG::2013-11-26 13:46:48,636::resourceManager::640::ResourceManager::(releaseResource) Resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c' is free, finding out if anyone is waiting for it.
>Thread-22::DEBUG::2013-11-26 13:46:48,636::resourceManager::648::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', Clearing records.
>Thread-22::DEBUG::2013-11-26 13:46:48,637::resourceManager::615::ResourceManager::(releaseResource) Trying to release resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34'
>Thread-22::DEBUG::2013-11-26 13:46:48,637::resourceManager::634::ResourceManager::(releaseResource) Released resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34' (0 active users)
>Thread-22::DEBUG::2013-11-26 13:46:48,637::resourceManager::640::ResourceManager::(releaseResource) Resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34' is free, finding out if anyone is waiting for it.
>Thread-22::DEBUG::2013-11-26 13:46:48,637::resourceManager::648::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34', Clearing records.
>Thread-22::DEBUG::2013-11-26 13:46:48,638::task::974::TaskManager.Task::(_decref) Task=`755ed18c-8b4f-448a-abb8-6e8a7be7d958`::ref 0 aborting False
>Thread-27::DEBUG::2013-11-26 13:46:48,716::BindingXMLRPC::177::vds::(wrapper) client [127.0.0.1]
>Thread-27::DEBUG::2013-11-26 13:46:48,716::task::579::TaskManager.Task::(_updateState) Task=`97f841d1-1f40-4028-bbe3-8d5166c3c446`::moving from state init -> state preparing
>Thread-27::INFO::2013-11-26 13:46:48,716::logUtils::44::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID='4f04507e-5611-4853-91ff-4384cabd8f34', hostID=1, scsiKey='4f04507e-5611-4853-91ff-4384cabd8f34', msdUUID='23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', masterVersion=1, options=None)
>Thread-27::DEBUG::2013-11-26 13:46:48,717::resourceManager::197::ResourceManager.Request::(__init__) ResName=`Storage.HsmDomainMonitorLock`ReqID=`c805a0cb-363f-43a7-a551-e8f38962171c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1012' at 'connectStoragePool'
>Thread-27::DEBUG::2013-11-26 13:46:48,717::resourceManager::541::ResourceManager::(registerResource) Trying to register resource 'Storage.HsmDomainMonitorLock' for lock type 'exclusive'
>Thread-27::DEBUG::2013-11-26 13:46:48,718::resourceManager::600::ResourceManager::(registerResource) Resource 'Storage.HsmDomainMonitorLock' is free. Now locking as 'exclusive' (1 active user)
>Thread-27::DEBUG::2013-11-26 13:46:48,718::resourceManager::237::ResourceManager.Request::(grant) ResName=`Storage.HsmDomainMonitorLock`ReqID=`c805a0cb-363f-43a7-a551-e8f38962171c`::Granted request
>Thread-27::DEBUG::2013-11-26 13:46:48,719::resourceManager::197::ResourceManager.Request::(__init__) ResName=`Storage.4f04507e-5611-4853-91ff-4384cabd8f34`ReqID=`a6276273-0bda-4e2d-a8e2-88df945e8c5d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1043' at '_connectStoragePool'
>Thread-27::DEBUG::2013-11-26 13:46:48,719::resourceManager::541::ResourceManager::(registerResource) Trying to register resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34' for lock type 'exclusive'
>Thread-27::DEBUG::2013-11-26 13:46:48,719::resourceManager::600::ResourceManager::(registerResource) Resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34' is free. Now locking as 'exclusive' (1 active user)
>Thread-27::DEBUG::2013-11-26 13:46:48,719::resourceManager::237::ResourceManager.Request::(grant) ResName=`Storage.4f04507e-5611-4853-91ff-4384cabd8f34`ReqID=`a6276273-0bda-4e2d-a8e2-88df945e8c5d`::Granted request
>Thread-27::INFO::2013-11-26 13:46:48,720::sp::678::Storage.StoragePool::(connect) Connect host #1 to the storage pool 4f04507e-5611-4853-91ff-4384cabd8f34 with master domain: 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c (ver = 1)
>Thread-27::DEBUG::2013-11-26 13:46:48,720::lvm::510::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-27::DEBUG::2013-11-26 13:46:48,721::lvm::512::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-27::DEBUG::2013-11-26 13:46:48,721::lvm::521::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-27::DEBUG::2013-11-26 13:46:48,721::lvm::523::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-27::DEBUG::2013-11-26 13:46:48,722::lvm::541::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-27::DEBUG::2013-11-26 13:46:48,722::lvm::543::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-27::DEBUG::2013-11-26 13:46:48,722::misc::809::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage)
>Thread-27::DEBUG::2013-11-26 13:46:48,722::misc::811::SamplingMethod::(__call__) Got in to sampling method
>Thread-27::DEBUG::2013-11-26 13:46:48,723::misc::809::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan)
>Thread-27::DEBUG::2013-11-26 13:46:48,723::misc::811::SamplingMethod::(__call__) Got in to sampling method
>Thread-27::DEBUG::2013-11-26 13:46:48,723::iscsiadm::91::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None)
>Thread-27::DEBUG::2013-11-26 13:46:48,746::iscsiadm::91::Storage.Misc.excCmd::(_runCmd) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21
>Thread-27::DEBUG::2013-11-26 13:46:48,748::misc::819::SamplingMethod::(__call__) Returning last result
>Thread-27::DEBUG::2013-11-26 13:46:53,771::multipath::112::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None)
>Thread-27::DEBUG::2013-11-26 13:46:53,899::multipath::112::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0
>Thread-27::DEBUG::2013-11-26 13:46:53,899::lvm::510::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-27::DEBUG::2013-11-26 13:46:53,900::lvm::512::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-27::DEBUG::2013-11-26 13:46:53,900::lvm::521::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-27::DEBUG::2013-11-26 13:46:53,900::lvm::523::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-27::DEBUG::2013-11-26 13:46:53,901::lvm::541::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-27::DEBUG::2013-11-26 13:46:53,901::lvm::543::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-27::DEBUG::2013-11-26 13:46:53,902::misc::819::SamplingMethod::(__call__) Returning last result
>Thread-27::DEBUG::2013-11-26 13:46:53,910::fileSD::154::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-27::DEBUG::2013-11-26 13:46:53,911::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend
>Thread-27::DEBUG::2013-11-26 13:46:53,921::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=4f04507e-5611-4853-91ff-4384cabd8f34', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine', 'ROLE=Master', 'SDUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=551213bd049cc5a04909832cf1427f157b080822']
>Thread-27::DEBUG::2013-11-26 13:46:53,923::fileSD::572::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images []
>Thread-27::INFO::2013-11-26 13:46:53,923::sd::374::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_imageNS already registered
>Thread-27::INFO::2013-11-26 13:46:53,923::sd::382::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_volumeNS already registered
>Thread-27::DEBUG::2013-11-26 13:46:53,924::sp::1533::Storage.StoragePool::(getMasterDomain) Master domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c verified, version 1
>Thread-27::DEBUG::2013-11-26 13:46:53,924::misc::809::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads)
>Thread-27::DEBUG::2013-11-26 13:46:53,924::misc::811::SamplingMethod::(__call__) Got in to sampling method
>Thread-27::DEBUG::2013-11-26 13:46:53,933::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=4f04507e-5611-4853-91ff-4384cabd8f34', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine', 'ROLE=Master', 'SDUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=551213bd049cc5a04909832cf1427f157b080822']
>Thread-31::DEBUG::2013-11-26 13:46:53,934::domainMonitor::159::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-27::DEBUG::2013-11-26 13:46:53,934::sp::1566::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `4f04507e-5611-4853-91ff-4384cabd8f34` started monitoring domain `23b22ad4-6d4c-4cdc-b25e-6e9efef7004c`
>Thread-27::DEBUG::2013-11-26 13:46:53,935::misc::819::SamplingMethod::(__call__) Returning last result
>Thread-27::WARNING::2013-11-26 13:46:53,937::fileUtils::167::Storage.fileUtils::(createdir) Dir /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34 already exists
>Thread-27::DEBUG::2013-11-26 13:46:53,938::lvm::386::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-27::DEBUG::2013-11-26 13:46:53,941::lvm::309::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:46:53,954::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:46:53,961::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n507 bytes (507 B) copied, 0.000453422 s, 1.1 MB/s\n'; <rc> = 0
>Thread-31::INFO::2013-11-26 13:46:53,964::clusterlock::174::SANLock::(acquireHostId) Acquiring host id for domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c (id: 1)
>Thread-31::DEBUG::2013-11-26 13:46:53,965::clusterlock::192::SANLock::(acquireHostId) Host id for domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c successfully acquired (id: 1)
>Thread-27::DEBUG::2013-11-26 13:46:54,172::lvm::309::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = '  No volume groups found\n'; <rc> = 0
>Thread-27::DEBUG::2013-11-26 13:46:54,172::lvm::428::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-27::INFO::2013-11-26 13:46:54,182::sp::1196::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c to /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-27::INFO::2013-11-26 13:46:54,183::sp::1196::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c to /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/mastersd
>Thread-27::DEBUG::2013-11-26 13:46:54,184::resourceManager::615::ResourceManager::(releaseResource) Trying to release resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34'
>Thread-27::DEBUG::2013-11-26 13:46:54,184::resourceManager::634::ResourceManager::(releaseResource) Released resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34' (0 active users)
>Thread-27::DEBUG::2013-11-26 13:46:54,184::resourceManager::640::ResourceManager::(releaseResource) Resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34' is free, finding out if anyone is waiting for it.
>Thread-27::DEBUG::2013-11-26 13:46:54,185::resourceManager::648::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34', Clearing records.
>Thread-27::DEBUG::2013-11-26 13:46:54,185::resourceManager::615::ResourceManager::(releaseResource) Trying to release resource 'Storage.HsmDomainMonitorLock'
>Thread-27::DEBUG::2013-11-26 13:46:54,185::resourceManager::634::ResourceManager::(releaseResource) Released resource 'Storage.HsmDomainMonitorLock' (0 active users)
>Thread-27::DEBUG::2013-11-26 13:46:54,186::resourceManager::640::ResourceManager::(releaseResource) Resource 'Storage.HsmDomainMonitorLock' is free, finding out if anyone is waiting for it.
>Thread-27::DEBUG::2013-11-26 13:46:54,186::resourceManager::648::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.HsmDomainMonitorLock', Clearing records.
>Thread-27::INFO::2013-11-26 13:46:54,186::logUtils::47::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True
>Thread-27::DEBUG::2013-11-26 13:46:54,187::task::1168::TaskManager.Task::(prepare) Task=`97f841d1-1f40-4028-bbe3-8d5166c3c446`::finished: True
>Thread-27::DEBUG::2013-11-26 13:46:54,187::task::579::TaskManager.Task::(_updateState) Task=`97f841d1-1f40-4028-bbe3-8d5166c3c446`::moving from state preparing -> state finished
>Thread-27::DEBUG::2013-11-26 13:46:54,187::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-27::DEBUG::2013-11-26 13:46:54,187::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-27::DEBUG::2013-11-26 13:46:54,188::task::974::TaskManager.Task::(_decref) Task=`97f841d1-1f40-4028-bbe3-8d5166c3c446`::ref 0 aborting False
>Thread-35::DEBUG::2013-11-26 13:46:54,244::BindingXMLRPC::177::vds::(wrapper) client [127.0.0.1]
>Thread-35::DEBUG::2013-11-26 13:46:54,244::task::579::TaskManager.Task::(_updateState) Task=`7460af63-41f5-48b7-9d3d-e299a3aeb05c`::moving from state init -> state preparing
>Thread-35::INFO::2013-11-26 13:46:54,245::logUtils::44::dispatcher::(wrapper) Run and protect: spmStart(spUUID='4f04507e-5611-4853-91ff-4384cabd8f34', prevID=-1, prevLVER=-1, recoveryMode=None, scsiFencing='false', maxHostID=250, domVersion=3, options=None)
>Thread-35::DEBUG::2013-11-26 13:46:54,245::resourceManager::197::ResourceManager.Request::(__init__) ResName=`Storage.4f04507e-5611-4853-91ff-4384cabd8f34`ReqID=`3d21374a-c302-40a2-a57d-f3352344ea7e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '586' at 'spmStart'
>Thread-35::DEBUG::2013-11-26 13:46:54,246::resourceManager::541::ResourceManager::(registerResource) Trying to register resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34' for lock type 'exclusive'
>Thread-35::DEBUG::2013-11-26 13:46:54,246::resourceManager::600::ResourceManager::(registerResource) Resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34' is free. Now locking as 'exclusive' (1 active user)
>Thread-35::DEBUG::2013-11-26 13:46:54,246::resourceManager::237::ResourceManager.Request::(grant) ResName=`Storage.4f04507e-5611-4853-91ff-4384cabd8f34`ReqID=`3d21374a-c302-40a2-a57d-f3352344ea7e`::Granted request
>Thread-35::DEBUG::2013-11-26 13:46:54,247::task::811::TaskManager.Task::(resourceAcquired) Task=`7460af63-41f5-48b7-9d3d-e299a3aeb05c`::_resourcesAcquired: Storage.4f04507e-5611-4853-91ff-4384cabd8f34 (exclusive)
>Thread-35::DEBUG::2013-11-26 13:46:54,247::task::974::TaskManager.Task::(_decref) Task=`7460af63-41f5-48b7-9d3d-e299a3aeb05c`::ref 1 aborting False
>Thread-35::INFO::2013-11-26 13:46:54,247::logUtils::47::dispatcher::(wrapper) Run and protect: spmStart, Return response: None
>Thread-35::DEBUG::2013-11-26 13:46:54,248::task::1163::TaskManager.Task::(prepare) Task=`7460af63-41f5-48b7-9d3d-e299a3aeb05c`::Prepare: 1 jobs exist, move to acquiring
>Thread-35::DEBUG::2013-11-26 13:46:54,248::task::579::TaskManager.Task::(_updateState) Task=`7460af63-41f5-48b7-9d3d-e299a3aeb05c`::moving from state preparing -> state acquiring
>Thread-35::DEBUG::2013-11-26 13:46:54,248::task::579::TaskManager.Task::(_updateState) Task=`7460af63-41f5-48b7-9d3d-e299a3aeb05c`::moving from state acquiring -> state queued
>Thread-35::DEBUG::2013-11-26 13:46:54,248::taskManager::50::TaskManager::(_queueTask) queuing task: 7460af63-41f5-48b7-9d3d-e299a3aeb05c
>Thread-35::DEBUG::2013-11-26 13:46:54,249::taskManager::56::TaskManager::(_queueTask) task queued: 7460af63-41f5-48b7-9d3d-e299a3aeb05c
>Thread-1::DEBUG::2013-11-26 13:46:54,249::threadPool::57::Misc.ThreadPool::(setRunningTask) Number of running tasks: 1
>Thread-35::DEBUG::2013-11-26 13:46:54,250::task::1165::TaskManager.Task::(prepare) Task=`7460af63-41f5-48b7-9d3d-e299a3aeb05c`::returning
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:46:54,250::threadPool::205::Misc.ThreadPool.WorkerThread::(run) Task: 7460af63-41f5-48b7-9d3d-e299a3aeb05c running: <bound method Task.commit of <storage.task.Task instance at 0x7fb0a0052098>> with: None
>Thread-35::DEBUG::2013-11-26 13:46:54,251::task::974::TaskManager.Task::(_decref) Task=`7460af63-41f5-48b7-9d3d-e299a3aeb05c`::ref 0 aborting False
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:46:54,252::task::1176::TaskManager.Task::(commit) Task=`7460af63-41f5-48b7-9d3d-e299a3aeb05c`::committing task: 7460af63-41f5-48b7-9d3d-e299a3aeb05c
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:46:54,253::task::579::TaskManager.Task::(_updateState) Task=`7460af63-41f5-48b7-9d3d-e299a3aeb05c`::moving from state queued -> state running
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:46:54,255::task::889::TaskManager.Task::(_runJobs) Task=`7460af63-41f5-48b7-9d3d-e299a3aeb05c`::Task.run: running job 0: spmStart: <bound method StoragePool.startSpm of <storage.sp.StoragePool object at 0x15a0dd0>> (args: (-1, -1, 'false', 250, 3) kwargs: {})
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:46:54,256::task::315::TaskManager.Task::(run) Task=`7460af63-41f5-48b7-9d3d-e299a3aeb05c`::Job.run: running spmStart: <bound method StoragePool.startSpm of <storage.sp.StoragePool object at 0x15a0dd0>> (args: (-1, -1, 'false', 250, 3) kwargs: {}) callback None
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:46:54,256::misc::809::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads)
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:46:54,256::misc::811::SamplingMethod::(__call__) Got in to sampling method
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:46:54,266::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=4f04507e-5611-4853-91ff-4384cabd8f34', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine', 'ROLE=Master', 'SDUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=551213bd049cc5a04909832cf1427f157b080822']
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:46:54,267::misc::819::SamplingMethod::(__call__) Returning last result
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:46:54,277::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=4f04507e-5611-4853-91ff-4384cabd8f34', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine', 'ROLE=Master', 'SDUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=551213bd049cc5a04909832cf1427f157b080822']
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::INFO::2013-11-26 13:46:54,277::clusterlock::174::SANLock::(acquireHostId) Acquiring host id for domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c (id: 1)
>Thread-36::DEBUG::2013-11-26 13:46:54,309::BindingXMLRPC::177::vds::(wrapper) client [127.0.0.1]
>Thread-36::DEBUG::2013-11-26 13:46:54,310::task::579::TaskManager.Task::(_updateState) Task=`d1a9376c-17ac-4de2-8090-9880c7b9bf50`::moving from state init -> state preparing
>Thread-36::INFO::2013-11-26 13:46:54,310::logUtils::44::dispatcher::(wrapper) Run and protect: activateStorageDomain(sdUUID='23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', spUUID='4f04507e-5611-4853-91ff-4384cabd8f34', options=None)
>Thread-36::DEBUG::2013-11-26 13:46:54,311::resourceManager::197::ResourceManager.Request::(__init__) ResName=`Storage.4f04507e-5611-4853-91ff-4384cabd8f34`ReqID=`26e054ba-fd24-4896-9e56-6308fca10577`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1234' at 'activateStorageDomain'
>Thread-36::DEBUG::2013-11-26 13:46:54,311::resourceManager::541::ResourceManager::(registerResource) Trying to register resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34' for lock type 'exclusive'
>Thread-36::DEBUG::2013-11-26 13:46:54,311::resourceManager::576::ResourceManager::(registerResource) Resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34' is currently locked, Entering queue (1 in queue)
>Thread-31::DEBUG::2013-11-26 13:47:03,978::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:47:03,985::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n507 bytes (507 B) copied, 0.000358442 s, 1.4 MB/s\n'; <rc> = 0
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,281::clusterlock::192::SANLock::(acquireHostId) Host id for domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c successfully acquired (id: 1)
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::INFO::2013-11-26 13:47:15,289::clusterlock::225::SANLock::(acquire) Acquiring cluster lock for domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c (id: 1)
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,351::clusterlock::251::SANLock::(acquire) Cluster lock for domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c successfully acquired (id: 1)
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,352::sp::276::Storage.StoragePool::(startSpm) spm lock acquired successfully
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,363::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=4f04507e-5611-4853-91ff-4384cabd8f34', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine', 'ROLE=Master', 'SDUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=551213bd049cc5a04909832cf1427f157b080822']
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,364::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,364::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,364::persistentDict::299::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=4f04507e-5611-4853-91ff-4384cabd8f34', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine', 'ROLE=Master', 'SDUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f301ce975088079aedbb732b24ddd897ad868d4d']
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,411::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,412::resourceManager::197::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_4f04507e-5611-4853-91ff-4384cabd8f34`ReqID=`2d5dcc87-7243-47d0-81d4-c403d80b4923`::Request was made in '/usr/share/vdsm/storage/sp.py' line '436' at '_upgradePool'
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,412::resourceManager::541::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_4f04507e-5611-4853-91ff-4384cabd8f34' for lock type 'exclusive'
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,413::resourceManager::600::ResourceManager::(registerResource) Resource 'Storage.upgrade_4f04507e-5611-4853-91ff-4384cabd8f34' is free. Now locking as 'exclusive' (1 active user)
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,413::resourceManager::237::ResourceManager.Request::(grant) ResName=`Storage.upgrade_4f04507e-5611-4853-91ff-4384cabd8f34`ReqID=`2d5dcc87-7243-47d0-81d4-c403d80b4923`::Granted request
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::INFO::2013-11-26 13:47:15,414::sp::442::Storage.StoragePool::(_upgradePool) Trying to upgrade master domain `23b22ad4-6d4c-4cdc-b25e-6e9efef7004c`
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,414::resourceManager::197::ResourceManager.Request::(__init__) ResName=`Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c`ReqID=`c2e3e5fb-6196-475c-9f8e-a2037af5a32a`::Request was made in '/usr/share/vdsm/storage/sp.py' line '444' at '_upgradePool'
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,414::resourceManager::541::ResourceManager::(registerResource) Trying to register resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c' for lock type 'exclusive'
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,415::resourceManager::600::ResourceManager::(registerResource) Resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c' is free. Now locking as 'exclusive' (1 active user)
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,415::resourceManager::237::ResourceManager.Request::(grant) ResName=`Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c`ReqID=`c2e3e5fb-6196-475c-9f8e-a2037af5a32a`::Granted request
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,428::fileSD::154::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,429::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,439::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=4f04507e-5611-4853-91ff-4384cabd8f34', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine', 'ROLE=Master', 'SDUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f301ce975088079aedbb732b24ddd897ad868d4d']
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,441::fileSD::572::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images []
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::INFO::2013-11-26 13:47:15,441::sd::374::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_imageNS already registered
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::INFO::2013-11-26 13:47:15,441::sd::382::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_volumeNS already registered
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,442::resourceManager::615::ResourceManager::(releaseResource) Trying to release resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c'
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,442::resourceManager::634::ResourceManager::(releaseResource) Released resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c' (0 active users)
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,442::resourceManager::640::ResourceManager::(releaseResource) Resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c' is free, finding out if anyone is waiting for it.
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,442::resourceManager::648::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', Clearing records.
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,443::sp::447::Storage.StoragePool::(_upgradePool) Marking all domains for upgrade
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,443::sp::454::Storage.StoragePool::(_upgradePool) Registering with state change event
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,443::sp::457::Storage.StoragePool::(_upgradePool) Running initial domain upgrade threads
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,444::resourceManager::615::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_4f04507e-5611-4853-91ff-4384cabd8f34'
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,444::resourceManager::634::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_4f04507e-5611-4853-91ff-4384cabd8f34' (0 active users)
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,444::resourceManager::640::ResourceManager::(releaseResource) Resource 'Storage.upgrade_4f04507e-5611-4853-91ff-4384cabd8f34' is free, finding out if anyone is waiting for it.
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,445::resourceManager::648::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_4f04507e-5611-4853-91ff-4384cabd8f34', Clearing records.
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,447::sd::620::Storage.StorageDomain::(createMasterTree) creating vms dir: /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/master/vms
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,461::sd::623::Storage.StorageDomain::(createMasterTree) creating task dir: /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/master/tasks
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,463::sp::337::Storage.StoragePool::(startSpm) ended.
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,464::task::579::TaskManager.Task::(_updateState) Task=`7460af63-41f5-48b7-9d3d-e299a3aeb05c`::moving from state running -> state finished
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,464::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4f04507e-5611-4853-91ff-4384cabd8f34': < ResourceRef 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34', isValid: 'True' obj: 'None'>}
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,464::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,465::resourceManager::615::ResourceManager::(releaseResource) Trying to release resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34'
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,465::resourceManager::634::ResourceManager::(releaseResource) Released resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34' (0 active users)
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,465::resourceManager::640::ResourceManager::(releaseResource) Resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34' is free, finding out if anyone is waiting for it.
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,465::resourceManager::653::ResourceManager::(releaseResource) Resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34' has 1 requests in queue. Handling top request.
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,466::resourceManager::237::ResourceManager.Request::(grant) ResName=`Storage.4f04507e-5611-4853-91ff-4384cabd8f34`ReqID=`26e054ba-fd24-4896-9e56-6308fca10577`::Granted request
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,466::resourceManager::685::ResourceManager::(releaseResource) Request 'Request for Storage.4f04507e-5611-4853-91ff-4384cabd8f34 - exclusive: granted' was granted
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,468::task::897::TaskManager.Task::(_runJobs) Task=`7460af63-41f5-48b7-9d3d-e299a3aeb05c`::Task.run: exit - success: result 
>Thread-36::DEBUG::2013-11-26 13:47:15,468::task::811::TaskManager.Task::(resourceAcquired) Task=`d1a9376c-17ac-4de2-8090-9880c7b9bf50`::_resourcesAcquired: Storage.4f04507e-5611-4853-91ff-4384cabd8f34 (exclusive)
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,469::task::974::TaskManager.Task::(_decref) Task=`7460af63-41f5-48b7-9d3d-e299a3aeb05c`::ref 0 aborting False
>Thread-36::DEBUG::2013-11-26 13:47:15,470::task::974::TaskManager.Task::(_decref) Task=`d1a9376c-17ac-4de2-8090-9880c7b9bf50`::ref 1 aborting False
>7460af63-41f5-48b7-9d3d-e299a3aeb05c::DEBUG::2013-11-26 13:47:15,470::threadPool::57::Misc.ThreadPool::(setRunningTask) Number of running tasks: 0
>Thread-36::DEBUG::2013-11-26 13:47:15,472::resourceManager::197::ResourceManager.Request::(__init__) ResName=`Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c`ReqID=`e11f735b-cd07-4672-a3a0-570d154559ea`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1235' at 'activateStorageDomain'
>Thread-36::DEBUG::2013-11-26 13:47:15,472::resourceManager::541::ResourceManager::(registerResource) Trying to register resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c' for lock type 'exclusive'
>Thread-36::DEBUG::2013-11-26 13:47:15,472::resourceManager::600::ResourceManager::(registerResource) Resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c' is free. Now locking as 'exclusive' (1 active user)
>Thread-36::DEBUG::2013-11-26 13:47:15,473::resourceManager::237::ResourceManager.Request::(grant) ResName=`Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c`ReqID=`e11f735b-cd07-4672-a3a0-570d154559ea`::Granted request
>Thread-36::DEBUG::2013-11-26 13:47:15,473::task::811::TaskManager.Task::(resourceAcquired) Task=`d1a9376c-17ac-4de2-8090-9880c7b9bf50`::_resourcesAcquired: Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c (exclusive)
>Thread-36::DEBUG::2013-11-26 13:47:15,473::task::974::TaskManager.Task::(_decref) Task=`d1a9376c-17ac-4de2-8090-9880c7b9bf50`::ref 1 aborting False
>Thread-36::INFO::2013-11-26 13:47:15,474::sp::1110::Storage.StoragePool::(activateSD) sdUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c spUUID=4f04507e-5611-4853-91ff-4384cabd8f34
>Thread-36::INFO::2013-11-26 13:47:15,474::logUtils::47::dispatcher::(wrapper) Run and protect: activateStorageDomain, Return response: None
>Thread-36::DEBUG::2013-11-26 13:47:15,474::task::1168::TaskManager.Task::(prepare) Task=`d1a9376c-17ac-4de2-8090-9880c7b9bf50`::finished: None
>Thread-36::DEBUG::2013-11-26 13:47:15,475::task::579::TaskManager.Task::(_updateState) Task=`d1a9376c-17ac-4de2-8090-9880c7b9bf50`::moving from state preparing -> state finished
>Thread-36::DEBUG::2013-11-26 13:47:15,475::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c': < ResourceRef 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', isValid: 'True' obj: 'None'>, 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34': < ResourceRef 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34', isValid: 'True' obj: 'None'>}
>Thread-36::DEBUG::2013-11-26 13:47:15,475::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-36::DEBUG::2013-11-26 13:47:15,476::resourceManager::615::ResourceManager::(releaseResource) Trying to release resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c'
>Thread-36::DEBUG::2013-11-26 13:47:15,476::resourceManager::634::ResourceManager::(releaseResource) Released resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c' (0 active users)
>Thread-36::DEBUG::2013-11-26 13:47:15,476::resourceManager::640::ResourceManager::(releaseResource) Resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c' is free, finding out if anyone is waiting for it.
>Thread-36::DEBUG::2013-11-26 13:47:15,476::resourceManager::648::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', Clearing records.
>Thread-36::DEBUG::2013-11-26 13:47:15,477::resourceManager::615::ResourceManager::(releaseResource) Trying to release resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34'
>Thread-36::DEBUG::2013-11-26 13:47:15,477::resourceManager::634::ResourceManager::(releaseResource) Released resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34' (0 active users)
>Thread-36::DEBUG::2013-11-26 13:47:15,477::resourceManager::640::ResourceManager::(releaseResource) Resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34' is free, finding out if anyone is waiting for it.
>Thread-36::DEBUG::2013-11-26 13:47:15,478::resourceManager::648::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34', Clearing records.
>Thread-36::DEBUG::2013-11-26 13:47:15,478::task::974::TaskManager.Task::(_decref) Task=`d1a9376c-17ac-4de2-8090-9880c7b9bf50`::ref 0 aborting False
>Thread-40::DEBUG::2013-11-26 13:47:15,546::BindingXMLRPC::177::vds::(wrapper) client [127.0.0.1]
>Thread-40::DEBUG::2013-11-26 13:47:15,547::task::579::TaskManager.Task::(_updateState) Task=`208916b6-4a0c-46a7-9880-260fb832ef5f`::moving from state init -> state preparing
>Thread-40::INFO::2013-11-26 13:47:15,547::logUtils::44::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None)
>Thread-40::DEBUG::2013-11-26 13:47:15,548::taskManager::102::TaskManager::(getAllTasksStatuses) Entry.
>Thread-40::DEBUG::2013-11-26 13:47:15,548::taskManager::113::TaskManager::(getAllTasksStatuses) Return: {}
>Thread-40::INFO::2013-11-26 13:47:15,548::logUtils::47::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {}}
>Thread-40::DEBUG::2013-11-26 13:47:15,548::task::1168::TaskManager.Task::(prepare) Task=`208916b6-4a0c-46a7-9880-260fb832ef5f`::finished: {'allTasksStatus': {}}
>Thread-40::DEBUG::2013-11-26 13:47:15,549::task::579::TaskManager.Task::(_updateState) Task=`208916b6-4a0c-46a7-9880-260fb832ef5f`::moving from state preparing -> state finished
>Thread-40::DEBUG::2013-11-26 13:47:15,549::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-40::DEBUG::2013-11-26 13:47:15,549::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-40::DEBUG::2013-11-26 13:47:15,549::task::974::TaskManager.Task::(_decref) Task=`208916b6-4a0c-46a7-9880-260fb832ef5f`::ref 0 aborting False
>Thread-41::DEBUG::2013-11-26 13:47:15,604::BindingXMLRPC::177::vds::(wrapper) client [127.0.0.1]
>Thread-41::DEBUG::2013-11-26 13:47:15,605::task::579::TaskManager.Task::(_updateState) Task=`992726e4-00ac-4bfe-9580-736ed614fe11`::moving from state init -> state preparing
>Thread-41::INFO::2013-11-26 13:47:15,605::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4f04507e-5611-4853-91ff-4384cabd8f34', options=None)
>Thread-41::INFO::2013-11-26 13:47:15,606::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}}
>Thread-41::DEBUG::2013-11-26 13:47:15,606::task::1168::TaskManager.Task::(prepare) Task=`992726e4-00ac-4bfe-9580-736ed614fe11`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}}
>Thread-41::DEBUG::2013-11-26 13:47:15,606::task::579::TaskManager.Task::(_updateState) Task=`992726e4-00ac-4bfe-9580-736ed614fe11`::moving from state preparing -> state finished
>Thread-41::DEBUG::2013-11-26 13:47:15,606::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-41::DEBUG::2013-11-26 13:47:15,607::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-41::DEBUG::2013-11-26 13:47:15,607::task::974::TaskManager.Task::(_decref) Task=`992726e4-00ac-4bfe-9580-736ed614fe11`::ref 0 aborting False
>Thread-42::DEBUG::2013-11-26 13:47:15,661::BindingXMLRPC::177::vds::(wrapper) client [127.0.0.1]
>Thread-42::DEBUG::2013-11-26 13:47:15,662::task::579::TaskManager.Task::(_updateState) Task=`792e46e1-b5ce-4eb0-ab9f-396607657f25`::moving from state init -> state preparing
>Thread-42::INFO::2013-11-26 13:47:15,662::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4f04507e-5611-4853-91ff-4384cabd8f34', options=None)
>Thread-42::DEBUG::2013-11-26 13:47:15,663::resourceManager::197::ResourceManager.Request::(__init__) ResName=`Storage.4f04507e-5611-4853-91ff-4384cabd8f34`ReqID=`514707eb-d4fa-4c23-aba2-ac0bd54601f2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2557' at 'getStoragePoolInfo'
>Thread-42::DEBUG::2013-11-26 13:47:15,663::resourceManager::541::ResourceManager::(registerResource) Trying to register resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34' for lock type 'shared'
>Thread-42::DEBUG::2013-11-26 13:47:15,663::resourceManager::600::ResourceManager::(registerResource) Resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34' is free. Now locking as 'shared' (1 active user)
>Thread-42::DEBUG::2013-11-26 13:47:15,664::resourceManager::237::ResourceManager.Request::(grant) ResName=`Storage.4f04507e-5611-4853-91ff-4384cabd8f34`ReqID=`514707eb-d4fa-4c23-aba2-ac0bd54601f2`::Granted request
>Thread-42::DEBUG::2013-11-26 13:47:15,664::task::811::TaskManager.Task::(resourceAcquired) Task=`792e46e1-b5ce-4eb0-ab9f-396607657f25`::_resourcesAcquired: Storage.4f04507e-5611-4853-91ff-4384cabd8f34 (shared)
>Thread-42::DEBUG::2013-11-26 13:47:15,664::task::974::TaskManager.Task::(_decref) Task=`792e46e1-b5ce-4eb0-ab9f-396607657f25`::ref 1 aborting False
>Thread-42::INFO::2013-11-26 13:47:15,669::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', 'name': 'hosted_datacenter', 'version': '3', 'domains': '23b22ad4-6d4c-4cdc-b25e-6e9efef7004c:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'23b22ad4-6d4c-4cdc-b25e-6e9efef7004c': {'status': 'Active', 'diskfree': '302563459072', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}}}
>Thread-42::DEBUG::2013-11-26 13:47:15,670::task::1168::TaskManager.Task::(prepare) Task=`792e46e1-b5ce-4eb0-ab9f-396607657f25`::finished: {'info': {'spm_id': 1, 'master_uuid': '23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', 'name': 'hosted_datacenter', 'version': '3', 'domains': '23b22ad4-6d4c-4cdc-b25e-6e9efef7004c:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'23b22ad4-6d4c-4cdc-b25e-6e9efef7004c': {'status': 'Active', 'diskfree': '302563459072', 'isoprefix': '', 'alerts': [], 'disktotal': '422758580224', 'version': 3}}}
>Thread-42::DEBUG::2013-11-26 13:47:15,670::task::579::TaskManager.Task::(_updateState) Task=`792e46e1-b5ce-4eb0-ab9f-396607657f25`::moving from state preparing -> state finished
>Thread-42::DEBUG::2013-11-26 13:47:15,670::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4f04507e-5611-4853-91ff-4384cabd8f34': < ResourceRef 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34', isValid: 'True' obj: 'None'>}
>Thread-42::DEBUG::2013-11-26 13:47:15,670::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-42::DEBUG::2013-11-26 13:47:15,671::resourceManager::615::ResourceManager::(releaseResource) Trying to release resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34'
>Thread-42::DEBUG::2013-11-26 13:47:15,671::resourceManager::634::ResourceManager::(releaseResource) Released resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34' (0 active users)
>Thread-42::DEBUG::2013-11-26 13:47:15,671::resourceManager::640::ResourceManager::(releaseResource) Resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34' is free, finding out if anyone is waiting for it.
>Thread-42::DEBUG::2013-11-26 13:47:15,672::resourceManager::648::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34', Clearing records.
>Thread-42::DEBUG::2013-11-26 13:47:15,672::task::974::TaskManager.Task::(_decref) Task=`792e46e1-b5ce-4eb0-ab9f-396607657f25`::ref 0 aborting False
>Thread-43::DEBUG::2013-11-26 13:47:15,728::BindingXMLRPC::177::vds::(wrapper) client [127.0.0.1]
>Thread-43::DEBUG::2013-11-26 13:47:15,729::task::579::TaskManager.Task::(_updateState) Task=`c6e311c5-3818-4a50-97d8-318455dbc808`::moving from state init -> state preparing
>Thread-43::INFO::2013-11-26 13:47:15,729::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-43::INFO::2013-11-26 13:47:15,730::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'23b22ad4-6d4c-4cdc-b25e-6e9efef7004c': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000358442', 'lastCheck': '0.4', 'valid': True}}
>Thread-43::DEBUG::2013-11-26 13:47:15,730::task::1168::TaskManager.Task::(prepare) Task=`c6e311c5-3818-4a50-97d8-318455dbc808`::finished: {'23b22ad4-6d4c-4cdc-b25e-6e9efef7004c': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000358442', 'lastCheck': '0.4', 'valid': True}}
>Thread-43::DEBUG::2013-11-26 13:47:15,730::task::579::TaskManager.Task::(_updateState) Task=`c6e311c5-3818-4a50-97d8-318455dbc808`::moving from state preparing -> state finished
>Thread-43::DEBUG::2013-11-26 13:47:15,730::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-43::DEBUG::2013-11-26 13:47:15,731::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-43::DEBUG::2013-11-26 13:47:15,731::task::974::TaskManager.Task::(_decref) Task=`c6e311c5-3818-4a50-97d8-318455dbc808`::ref 0 aborting False
>Thread-44::DEBUG::2013-11-26 13:47:16,221::BindingXMLRPC::177::vds::(wrapper) client [127.0.0.1]
>Thread-44::DEBUG::2013-11-26 13:47:16,222::task::579::TaskManager.Task::(_updateState) Task=`264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8`::moving from state init -> state preparing
>Thread-44::INFO::2013-11-26 13:47:16,222::logUtils::44::dispatcher::(wrapper) Run and protect: createVolume(sdUUID='23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', spUUID='4f04507e-5611-4853-91ff-4384cabd8f34', imgUUID='9fe1b677-08cb-4f35-8eec-d22e199b357f', size=41943040, volFormat=5, preallocate=2, diskType=2, volUUID='e1f754c5-49b1-4d6f-a738-5ed24f712ee9', desc='rhel65_Disk1', srcImgUUID='00000000-0000-0000-0000-000000000000', srcVolUUID='00000000-0000-0000-0000-000000000000')
>Thread-44::DEBUG::2013-11-26 13:47:16,223::resourceManager::197::ResourceManager.Request::(__init__) ResName=`Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c`ReqID=`5e165a23-bd12-4ce3-9b27-331e7fed8239`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1479' at 'createVolume'
>Thread-44::DEBUG::2013-11-26 13:47:16,223::resourceManager::541::ResourceManager::(registerResource) Trying to register resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c' for lock type 'shared'
>Thread-44::DEBUG::2013-11-26 13:47:16,223::resourceManager::600::ResourceManager::(registerResource) Resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c' is free. Now locking as 'shared' (1 active user)
>Thread-44::DEBUG::2013-11-26 13:47:16,224::resourceManager::237::ResourceManager.Request::(grant) ResName=`Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c`ReqID=`5e165a23-bd12-4ce3-9b27-331e7fed8239`::Granted request
>Thread-44::DEBUG::2013-11-26 13:47:16,224::task::811::TaskManager.Task::(resourceAcquired) Task=`264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8`::_resourcesAcquired: Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c (shared)
>Thread-44::DEBUG::2013-11-26 13:47:16,224::task::974::TaskManager.Task::(_decref) Task=`264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8`::ref 1 aborting False
>Thread-44::DEBUG::2013-11-26 13:47:16,266::task::736::TaskManager.Task::(_save) Task=`264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8`::_save: orig /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/mastersd/master/tasks/264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8 temp /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/mastersd/master/tasks/264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8.temp
>Thread-44::DEBUG::2013-11-26 13:47:16,348::taskManager::68::TaskManager::(scheduleJob) scheduled job createVolume for task 264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8 
>Thread-44::INFO::2013-11-26 13:47:16,348::logUtils::47::dispatcher::(wrapper) Run and protect: createVolume, Return response: None
>Thread-44::DEBUG::2013-11-26 13:47:16,348::task::1163::TaskManager.Task::(prepare) Task=`264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8`::Prepare: 1 jobs exist, move to acquiring
>Thread-44::DEBUG::2013-11-26 13:47:16,348::task::579::TaskManager.Task::(_updateState) Task=`264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8`::moving from state preparing -> state acquiring
>Thread-44::DEBUG::2013-11-26 13:47:16,349::task::736::TaskManager.Task::(_save) Task=`264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8`::_save: orig /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/mastersd/master/tasks/264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8 temp /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/mastersd/master/tasks/264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8.temp
>Thread-44::DEBUG::2013-11-26 13:47:16,461::task::579::TaskManager.Task::(_updateState) Task=`264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8`::moving from state acquiring -> state queued
>Thread-44::DEBUG::2013-11-26 13:47:16,462::task::736::TaskManager.Task::(_save) Task=`264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8`::_save: orig /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/mastersd/master/tasks/264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8 temp /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/mastersd/master/tasks/264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8.temp
>Thread-44::DEBUG::2013-11-26 13:47:16,655::taskManager::50::TaskManager::(_queueTask) queuing task: 264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8
>Thread-10::DEBUG::2013-11-26 13:47:16,655::threadPool::57::Misc.ThreadPool::(setRunningTask) Number of running tasks: 1
>Thread-44::DEBUG::2013-11-26 13:47:16,655::taskManager::56::TaskManager::(_queueTask) task queued: 264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:16,656::threadPool::205::Misc.ThreadPool.WorkerThread::(run) Task: 264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8 running: <bound method Task.commit of <storage.task.Task instance at 0x7fb0a00cc170>> with: None
>Thread-44::DEBUG::2013-11-26 13:47:16,657::task::1165::TaskManager.Task::(prepare) Task=`264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8`::returning
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:16,657::task::1176::TaskManager.Task::(commit) Task=`264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8`::committing task: 264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8
>Thread-44::DEBUG::2013-11-26 13:47:16,658::task::974::TaskManager.Task::(_decref) Task=`264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8`::ref 0 aborting False
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:16,659::task::579::TaskManager.Task::(_updateState) Task=`264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8`::moving from state queued -> state running
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:16,662::task::736::TaskManager.Task::(_save) Task=`264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8`::_save: orig /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/mastersd/master/tasks/264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8 temp /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/mastersd/master/tasks/264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8.temp
>Thread-45::DEBUG::2013-11-26 13:47:16,716::BindingXMLRPC::177::vds::(wrapper) client [127.0.0.1]
>Thread-45::DEBUG::2013-11-26 13:47:16,717::task::579::TaskManager.Task::(_updateState) Task=`9d462853-0315-443c-8ab9-053204660ad2`::moving from state init -> state preparing
>Thread-45::INFO::2013-11-26 13:47:16,717::logUtils::44::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None)
>Thread-45::DEBUG::2013-11-26 13:47:16,717::taskManager::102::TaskManager::(getAllTasksStatuses) Entry.
>Thread-45::DEBUG::2013-11-26 13:47:16,718::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8
>Thread-45::DEBUG::2013-11-26 13:47:16,718::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8'}
>Thread-45::DEBUG::2013-11-26 13:47:16,718::taskManager::113::TaskManager::(getAllTasksStatuses) Return: {'264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8'}}
>Thread-45::INFO::2013-11-26 13:47:16,719::logUtils::47::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {'264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8'}}}
>Thread-45::DEBUG::2013-11-26 13:47:16,719::task::1168::TaskManager.Task::(prepare) Task=`9d462853-0315-443c-8ab9-053204660ad2`::finished: {'allTasksStatus': {'264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8'}}}
>Thread-45::DEBUG::2013-11-26 13:47:16,719::task::579::TaskManager.Task::(_updateState) Task=`9d462853-0315-443c-8ab9-053204660ad2`::moving from state preparing -> state finished
>Thread-45::DEBUG::2013-11-26 13:47:16,719::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-45::DEBUG::2013-11-26 13:47:16,720::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-45::DEBUG::2013-11-26 13:47:16,720::task::974::TaskManager.Task::(_decref) Task=`9d462853-0315-443c-8ab9-053204660ad2`::ref 0 aborting False
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:16,876::task::889::TaskManager.Task::(_runJobs) Task=`264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8`::Task.run: running job 0: createVolume: <bound method StoragePool.createVolume of <storage.sp.StoragePool object at 0x15a0dd0>> (args: ('23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', '9fe1b677-08cb-4f35-8eec-d22e199b357f', 41943040, 5, 2, 2, 'e1f754c5-49b1-4d6f-a738-5ed24f712ee9', 'rhel65_Disk1', '00000000-0000-0000-0000-000000000000', '00000000-0000-0000-0000-000000000000') kwargs: {})
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:16,876::task::315::TaskManager.Task::(run) Task=`264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8`::Job.run: running createVolume: <bound method StoragePool.createVolume of <storage.sp.StoragePool object at 0x15a0dd0>> (args: ('23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', '9fe1b677-08cb-4f35-8eec-d22e199b357f', 41943040, 5, 2, 2, 'e1f754c5-49b1-4d6f-a738-5ed24f712ee9', 'rhel65_Disk1', '00000000-0000-0000-0000-000000000000', '00000000-0000-0000-0000-000000000000') kwargs: {}) callback None
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:16,877::resourceManager::197::ResourceManager.Request::(__init__) ResName=`23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_imageNS.9fe1b677-08cb-4f35-8eec-d22e199b357f`ReqID=`3ec35214-6139-46d7-899e-a8e93e47b046`::Request was made in '/usr/share/vdsm/storage/sp.py' line '1899' at 'createVolume'
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:16,877::resourceManager::541::ResourceManager::(registerResource) Trying to register resource '23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_imageNS.9fe1b677-08cb-4f35-8eec-d22e199b357f' for lock type 'exclusive'
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:16,879::resourceFactories::125::Storage.ResourcesFactories::(__getResourceCandidatesList) Image 9fe1b677-08cb-4f35-8eec-d22e199b357f does not exist in domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:16,879::resourceManager::600::ResourceManager::(registerResource) Resource '23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_imageNS.9fe1b677-08cb-4f35-8eec-d22e199b357f' is free. Now locking as 'exclusive' (1 active user)
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:16,879::resourceManager::237::ResourceManager.Request::(grant) ResName=`23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_imageNS.9fe1b677-08cb-4f35-8eec-d22e199b357f`ReqID=`3ec35214-6139-46d7-899e-a8e93e47b046`::Granted request
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::INFO::2013-11-26 13:47:16,880::image::123::Storage.Image::(create) Create placeholder /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/images/9fe1b677-08cb-4f35-8eec-d22e199b357f for image's volumes
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:16,881::task::736::TaskManager.Task::(_save) Task=`264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8`::_save: orig /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/mastersd/master/tasks/264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8 temp /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/mastersd/master/tasks/264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8.temp
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::INFO::2013-11-26 13:47:17,070::volume::446::Storage.Volume::(create) Creating volume e1f754c5-49b1-4d6f-a738-5ed24f712ee9
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:17,071::task::736::TaskManager.Task::(_save) Task=`264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8`::_save: orig /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/mastersd/master/tasks/264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8 temp /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/mastersd/master/tasks/264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8.temp
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:17,232::task::736::TaskManager.Task::(_save) Task=`264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8`::_save: orig /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/mastersd/master/tasks/264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8 temp /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/mastersd/master/tasks/264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8.temp
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::INFO::2013-11-26 13:47:17,562::fileVolume::151::Storage.Volume::(_create) Request to create RAW volume /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/images/9fe1b677-08cb-4f35-8eec-d22e199b357f/e1f754c5-49b1-4d6f-a738-5ed24f712ee9 with size = 41943040 sectors
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:17,622::task::736::TaskManager.Task::(_save) Task=`264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8`::_save: orig /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/mastersd/master/tasks/264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8 temp /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/mastersd/master/tasks/264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8.temp
>Thread-46::DEBUG::2013-11-26 13:47:17,776::BindingXMLRPC::177::vds::(wrapper) client [127.0.0.1]
>Thread-46::DEBUG::2013-11-26 13:47:17,777::task::579::TaskManager.Task::(_updateState) Task=`5c9139ed-7c71-4fca-af84-d89edb76da7a`::moving from state init -> state preparing
>Thread-46::INFO::2013-11-26 13:47:17,777::logUtils::44::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None)
>Thread-46::DEBUG::2013-11-26 13:47:17,778::taskManager::102::TaskManager::(getAllTasksStatuses) Entry.
>Thread-46::DEBUG::2013-11-26 13:47:17,778::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8
>Thread-46::DEBUG::2013-11-26 13:47:17,778::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8'}
>Thread-46::DEBUG::2013-11-26 13:47:17,778::taskManager::113::TaskManager::(getAllTasksStatuses) Return: {'264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8'}}
>Thread-46::INFO::2013-11-26 13:47:17,779::logUtils::47::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {'264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8'}}}
>Thread-46::DEBUG::2013-11-26 13:47:17,779::task::1168::TaskManager.Task::(prepare) Task=`5c9139ed-7c71-4fca-af84-d89edb76da7a`::finished: {'allTasksStatus': {'264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8'}}}
>Thread-46::DEBUG::2013-11-26 13:47:17,779::task::579::TaskManager.Task::(_updateState) Task=`5c9139ed-7c71-4fca-af84-d89edb76da7a`::moving from state preparing -> state finished
>Thread-46::DEBUG::2013-11-26 13:47:17,779::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-46::DEBUG::2013-11-26 13:47:17,780::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-46::DEBUG::2013-11-26 13:47:17,780::task::974::TaskManager.Task::(_decref) Task=`5c9139ed-7c71-4fca-af84-d89edb76da7a`::ref 0 aborting False
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:17,883::fileVolume::382::Storage.Volume::(newVolumeLease) Initializing volume lease volUUID=e1f754c5-49b1-4d6f-a738-5ed24f712ee9 sdUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c, metaId=('/rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/images/9fe1b677-08cb-4f35-8eec-d22e199b357f/e1f754c5-49b1-4d6f-a738-5ed24f712ee9',)
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:17,986::task::736::TaskManager.Task::(_save) Task=`264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8`::_save: orig /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/mastersd/master/tasks/264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8 temp /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/mastersd/master/tasks/264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8.temp
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:18,193::resourceManager::615::ResourceManager::(releaseResource) Trying to release resource '23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_imageNS.9fe1b677-08cb-4f35-8eec-d22e199b357f'
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:18,193::resourceManager::634::ResourceManager::(releaseResource) Released resource '23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_imageNS.9fe1b677-08cb-4f35-8eec-d22e199b357f' (0 active users)
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:18,194::resourceManager::640::ResourceManager::(releaseResource) Resource '23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_imageNS.9fe1b677-08cb-4f35-8eec-d22e199b357f' is free, finding out if anyone is waiting for it.
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:18,194::resourceManager::648::ResourceManager::(releaseResource) No one is waiting for resource '23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_imageNS.9fe1b677-08cb-4f35-8eec-d22e199b357f', Clearing records.
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:18,194::task::579::TaskManager.Task::(_updateState) Task=`264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8`::moving from state running -> state finished
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:18,195::task::736::TaskManager.Task::(_save) Task=`264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8`::_save: orig /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/mastersd/master/tasks/264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8 temp /rhev/data-center/4f04507e-5611-4853-91ff-4384cabd8f34/mastersd/master/tasks/264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8.temp
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:18,470::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c': < ResourceRef 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', isValid: 'True' obj: 'None'>}
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:18,470::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:18,470::resourceManager::615::ResourceManager::(releaseResource) Trying to release resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c'
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:18,471::resourceManager::634::ResourceManager::(releaseResource) Released resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c' (0 active users)
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:18,471::resourceManager::640::ResourceManager::(releaseResource) Resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c' is free, finding out if anyone is waiting for it.
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:18,471::resourceManager::648::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', Clearing records.
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:18,471::task::897::TaskManager.Task::(_runJobs) Task=`264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8`::Task.run: exit - success: result {'uuid': 'e1f754c5-49b1-4d6f-a738-5ed24f712ee9'}
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:18,472::task::974::TaskManager.Task::(_decref) Task=`264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8`::ref 0 aborting False
>264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8::DEBUG::2013-11-26 13:47:18,472::threadPool::57::Misc.ThreadPool::(setRunningTask) Number of running tasks: 0
>Thread-47::DEBUG::2013-11-26 13:47:18,836::BindingXMLRPC::177::vds::(wrapper) client [127.0.0.1]
>Thread-47::DEBUG::2013-11-26 13:47:18,837::task::579::TaskManager.Task::(_updateState) Task=`0bdf1128-6819-4a91-82d9-91270b1a5431`::moving from state init -> state preparing
>Thread-47::INFO::2013-11-26 13:47:18,837::logUtils::44::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None)
>Thread-47::DEBUG::2013-11-26 13:47:18,837::taskManager::102::TaskManager::(getAllTasksStatuses) Entry.
>Thread-47::DEBUG::2013-11-26 13:47:18,837::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8
>Thread-47::DEBUG::2013-11-26 13:47:18,838::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8'}
>Thread-47::DEBUG::2013-11-26 13:47:18,838::taskManager::113::TaskManager::(getAllTasksStatuses) Return: {'264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8'}}
>Thread-47::INFO::2013-11-26 13:47:18,838::logUtils::47::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {'264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8'}}}
>Thread-47::DEBUG::2013-11-26 13:47:18,838::task::1168::TaskManager.Task::(prepare) Task=`0bdf1128-6819-4a91-82d9-91270b1a5431`::finished: {'allTasksStatus': {'264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8'}}}
>Thread-47::DEBUG::2013-11-26 13:47:18,839::task::579::TaskManager.Task::(_updateState) Task=`0bdf1128-6819-4a91-82d9-91270b1a5431`::moving from state preparing -> state finished
>Thread-47::DEBUG::2013-11-26 13:47:18,839::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-47::DEBUG::2013-11-26 13:47:18,839::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-47::DEBUG::2013-11-26 13:47:18,839::task::974::TaskManager.Task::(_decref) Task=`0bdf1128-6819-4a91-82d9-91270b1a5431`::ref 0 aborting False
>Thread-48::DEBUG::2013-11-26 13:47:18,894::BindingXMLRPC::177::vds::(wrapper) client [127.0.0.1]
>Thread-48::DEBUG::2013-11-26 13:47:18,895::task::579::TaskManager.Task::(_updateState) Task=`a020665e-d034-46be-9837-c170234e9281`::moving from state init -> state preparing
>Thread-48::INFO::2013-11-26 13:47:18,895::logUtils::44::dispatcher::(wrapper) Run and protect: clearTask(taskID='264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8', spUUID=None, options=None)
>Thread-48::DEBUG::2013-11-26 13:47:18,895::taskManager::161::TaskManager::(clearTask) Entry. taskID: 264cfdb5-f2de-4fa5-b4d0-1bd8ad0082c8
>Thread-48::DEBUG::2013-11-26 13:47:18,921::taskManager::166::TaskManager::(clearTask) Return.
>Thread-48::INFO::2013-11-26 13:47:18,921::logUtils::47::dispatcher::(wrapper) Run and protect: clearTask, Return response: None
>Thread-48::DEBUG::2013-11-26 13:47:18,921::task::1168::TaskManager.Task::(prepare) Task=`a020665e-d034-46be-9837-c170234e9281`::finished: None
>Thread-48::DEBUG::2013-11-26 13:47:18,921::task::579::TaskManager.Task::(_updateState) Task=`a020665e-d034-46be-9837-c170234e9281`::moving from state preparing -> state finished
>Thread-48::DEBUG::2013-11-26 13:47:18,922::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-48::DEBUG::2013-11-26 13:47:18,922::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-48::DEBUG::2013-11-26 13:47:18,922::task::974::TaskManager.Task::(_decref) Task=`a020665e-d034-46be-9837-c170234e9281`::ref 0 aborting False
>Thread-31::DEBUG::2013-11-26 13:47:25,298::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:47:25,304::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000301364 s, 1.7 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:47:35,316::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:47:35,322::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000329073 s, 1.5 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:47:45,332::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:47:45,338::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000292709 s, 1.7 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:47:55,538::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:47:55,559::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.00565838 s, 89.2 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:48:05,573::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:48:05,618::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.0347677 s, 14.5 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:48:15,637::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:48:15,643::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000350684 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:48:25,655::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:48:25,662::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000383828 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:48:35,673::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:48:35,678::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000247944 s, 2.0 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:48:45,689::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:48:45,694::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000298851 s, 1.7 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:48:55,705::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:48:55,711::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000356844 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:49:05,804::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:49:05,809::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000224729 s, 2.2 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:49:15,821::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:49:15,827::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000265372 s, 1.9 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:49:25,838::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:49:25,856::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000350084 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:49:35,867::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:49:35,873::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000267903 s, 1.9 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:49:45,886::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:49:45,954::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.0540303 s, 9.3 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:49:56,014::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:49:56,020::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000368189 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:50:06,093::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:50:06,123::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000412309 s, 1.2 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:50:16,136::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:50:16,142::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000413101 s, 1.2 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:50:26,155::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:50:26,161::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000436787 s, 1.2 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:50:36,171::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:50:36,177::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000338848 s, 1.5 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:50:46,189::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:50:46,194::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000362654 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:50:56,206::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:50:56,212::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000369021 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:51:06,225::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:51:06,231::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000400233 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:51:16,245::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:51:16,252::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000464969 s, 1.1 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:51:26,263::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:51:26,269::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000351283 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:51:36,280::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:51:36,285::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000350163 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:51:46,296::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:51:46,302::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000375781 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:51:56,305::domainMonitor::186::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-31::DEBUG::2013-11-26 13:51:56,310::fileSD::154::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-31::DEBUG::2013-11-26 13:51:56,311::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend
>Thread-31::DEBUG::2013-11-26 13:51:56,319::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=4f04507e-5611-4853-91ff-4384cabd8f34', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine', 'ROLE=Master', 'SDUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f301ce975088079aedbb732b24ddd897ad868d4d']
>Thread-31::DEBUG::2013-11-26 13:51:56,321::fileSD::572::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images []
>Thread-31::INFO::2013-11-26 13:51:56,322::sd::374::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_imageNS already registered
>Thread-31::INFO::2013-11-26 13:51:56,322::sd::382::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_volumeNS already registered
>Thread-31::DEBUG::2013-11-26 13:51:56,329::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:51:56,335::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000468613 s, 1.1 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:52:06,347::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:52:06,353::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000431111 s, 1.2 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:52:16,364::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:52:16,370::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000378684 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:52:26,380::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:52:26,386::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000402364 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:52:36,396::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:52:36,402::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000388443 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:52:46,412::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:52:46,419::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000383512 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:52:56,429::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:52:56,435::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000403561 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:53:06,445::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:53:06,451::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000393132 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:53:16,462::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:53:16,468::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000419159 s, 1.2 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:53:26,479::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:53:26,485::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000373325 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:53:36,495::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:53:36,501::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000278873 s, 1.8 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:53:46,511::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:53:46,517::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000313858 s, 1.6 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:53:56,527::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:53:56,532::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000454782 s, 1.1 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:54:06,543::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:54:06,549::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000340024 s, 1.5 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:54:16,560::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:54:16,566::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000426538 s, 1.2 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:54:26,576::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:54:26,582::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000375652 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:54:36,593::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:54:36,598::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000333073 s, 1.5 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:54:46,608::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:54:46,614::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000335443 s, 1.5 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:54:56,625::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:54:56,630::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000322975 s, 1.6 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:55:06,641::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:55:06,648::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000353853 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:55:16,658::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:55:16,664::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000426993 s, 1.2 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:55:26,675::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:55:26,681::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000317791 s, 1.6 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:55:36,691::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:55:36,697::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.00027937 s, 1.8 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:55:46,707::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:55:46,713::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000419958 s, 1.2 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:55:56,724::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:55:56,729::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000351018 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:56:06,741::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:56:06,746::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000352565 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:56:16,757::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:56:16,763::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000349329 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:56:26,773::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:56:26,779::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000305531 s, 1.7 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:56:36,789::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:56:36,795::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000437264 s, 1.2 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:56:46,806::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:56:46,811::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000372047 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:56:56,815::domainMonitor::186::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-31::DEBUG::2013-11-26 13:56:56,819::fileSD::154::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-31::DEBUG::2013-11-26 13:56:56,820::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend
>Thread-31::DEBUG::2013-11-26 13:56:56,827::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=4f04507e-5611-4853-91ff-4384cabd8f34', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine', 'ROLE=Master', 'SDUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f301ce975088079aedbb732b24ddd897ad868d4d']
>Thread-31::DEBUG::2013-11-26 13:56:56,828::fileSD::572::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images []
>Thread-31::INFO::2013-11-26 13:56:56,829::sd::374::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_imageNS already registered
>Thread-31::INFO::2013-11-26 13:56:56,829::sd::382::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_volumeNS already registered
>Thread-31::DEBUG::2013-11-26 13:56:56,836::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:56:56,841::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000454357 s, 1.1 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:57:06,852::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:57:06,858::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000375528 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:57:16,869::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:57:16,875::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000981703 s, 514 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:57:26,886::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:57:26,891::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000359712 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:57:36,902::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:57:36,907::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.0003992 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:57:46,918::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:57:46,923::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000455511 s, 1.1 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:57:56,934::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:57:56,939::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000383156 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:58:06,950::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:58:06,956::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000331693 s, 1.5 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:58:16,966::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:58:16,972::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000372153 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:58:26,982::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:58:26,988::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000308024 s, 1.6 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:58:36,998::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:58:37,004::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000353037 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:58:47,014::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:58:47,020::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000430704 s, 1.2 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:58:57,031::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:58:57,037::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000379253 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:59:07,048::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:59:07,054::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000321746 s, 1.6 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:59:17,064::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:59:17,070::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000393688 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:59:27,081::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:59:27,087::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000401907 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:59:37,097::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:59:37,103::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000368919 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:59:47,113::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:59:47,119::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000428633 s, 1.2 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 13:59:57,135::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 13:59:57,140::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000340398 s, 1.5 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:00:07,160::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:00:07,166::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000362202 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:00:17,186::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:00:17,192::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.00115465 s, 437 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:00:27,211::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:00:27,216::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000347643 s, 1.5 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:00:37,235::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:00:37,241::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000358918 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:00:47,260::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:00:47,266::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000311824 s, 1.6 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:00:57,285::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:00:57,291::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000370295 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:01:07,310::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:01:07,316::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000361699 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:01:17,336::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:01:17,342::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000317998 s, 1.6 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:01:27,361::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:01:27,367::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000289573 s, 1.7 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:01:37,386::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:01:37,392::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000316247 s, 1.6 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:01:47,412::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:01:47,418::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000460953 s, 1.1 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:01:57,422::domainMonitor::186::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-31::DEBUG::2013-11-26 14:01:57,435::fileSD::154::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-31::DEBUG::2013-11-26 14:01:57,436::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend
>Thread-31::DEBUG::2013-11-26 14:01:57,443::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=4f04507e-5611-4853-91ff-4384cabd8f34', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine', 'ROLE=Master', 'SDUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f301ce975088079aedbb732b24ddd897ad868d4d']
>Thread-31::DEBUG::2013-11-26 14:01:57,444::fileSD::572::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images []
>Thread-31::INFO::2013-11-26 14:01:57,444::sd::374::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_imageNS already registered
>Thread-31::INFO::2013-11-26 14:01:57,445::sd::382::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_volumeNS already registered
>Thread-31::DEBUG::2013-11-26 14:01:57,452::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:01:57,458::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000305706 s, 1.7 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:02:07,468::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:02:07,474::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000411735 s, 1.2 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:02:17,485::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:02:17,491::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000385928 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:02:27,509::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:02:27,515::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000398911 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:02:37,534::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:02:37,540::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000347361 s, 1.5 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:02:47,560::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:02:47,565::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000416742 s, 1.2 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:02:57,587::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:02:57,592::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000271675 s, 1.9 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:03:07,612::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:03:07,617::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000230558 s, 2.2 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:03:17,638::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:03:17,644::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000996534 s, 507 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:03:27,663::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:03:27,669::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000486445 s, 1.0 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:03:37,687::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:03:37,693::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.00036186 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:03:47,712::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:03:47,718::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000371239 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:03:57,738::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:03:57,744::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.00036661 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:04:07,763::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:04:07,769::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000337249 s, 1.5 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:04:17,789::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:04:17,795::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000460017 s, 1.1 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:04:27,920::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:04:28,035::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.076052 s, 6.6 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:04:38,075::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:04:38,081::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000329835 s, 1.5 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:04:48,091::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:04:48,097::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000454783 s, 1.1 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:04:58,109::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:04:58,115::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.00035994 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:05:08,127::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:05:08,133::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000367991 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:05:18,144::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:05:18,151::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000368745 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:05:28,161::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:05:28,167::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000347264 s, 1.5 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:05:38,177::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:05:38,183::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.00041118 s, 1.2 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:05:48,193::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:05:48,199::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000347727 s, 1.5 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:05:58,210::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:05:58,216::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.00035238 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:06:08,227::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:06:08,233::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000286674 s, 1.8 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:06:18,243::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:06:18,249::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000425933 s, 1.2 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:06:28,259::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:06:28,266::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000384698 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:06:38,276::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:06:38,282::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000482356 s, 1.0 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:06:48,292::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:06:48,298::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.00028326 s, 1.8 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:06:58,301::domainMonitor::186::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-31::DEBUG::2013-11-26 14:06:58,306::fileSD::154::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-31::DEBUG::2013-11-26 14:06:58,307::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend
>Thread-31::DEBUG::2013-11-26 14:06:58,313::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=4f04507e-5611-4853-91ff-4384cabd8f34', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine', 'ROLE=Master', 'SDUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f301ce975088079aedbb732b24ddd897ad868d4d']
>Thread-31::DEBUG::2013-11-26 14:06:58,314::fileSD::572::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images []
>Thread-31::INFO::2013-11-26 14:06:58,315::sd::374::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_imageNS already registered
>Thread-31::INFO::2013-11-26 14:06:58,315::sd::382::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_volumeNS already registered
>Thread-31::DEBUG::2013-11-26 14:06:58,322::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:06:58,328::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000355012 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:07:08,339::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:07:08,346::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.0003724 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:07:18,357::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:07:19,337::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000279819 s, 1.8 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:07:29,394::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:07:31,633::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000416312 s, 1.2 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:07:41,643::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:07:43,311::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000314472 s, 1.6 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:07:53,322::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:07:54,731::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.00129502 s, 390 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:08:04,742::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:08:04,748::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000306587 s, 1.6 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:08:14,763::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:08:16,950::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.00117643 s, 429 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:08:26,964::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:08:31,634::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000223779 s, 2.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:08:41,649::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:08:41,655::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000290443 s, 1.7 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:08:51,667::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:08:51,699::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000338308 s, 1.5 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:09:01,711::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:09:04,317::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000373136 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:09:14,329::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:09:15,707::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000314133 s, 1.6 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:09:25,720::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:09:25,726::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000388213 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:09:35,773::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:09:36,654::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000368984 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:09:46,665::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:09:46,671::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000495109 s, 1.0 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:09:56,685::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:09:58,136::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000462811 s, 1.1 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:10:08,149::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:10:08,730::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000336761 s, 1.5 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:10:18,744::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:10:18,869::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000350479 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:10:28,887::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:10:28,893::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000342486 s, 1.5 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:10:38,909::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:10:40,832::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000353037 s, 1.4 MB/s\n'; <rc> = 0
>Thread-61::DEBUG::2013-11-26 14:10:48,234::BindingXMLRPC::177::vds::(wrapper) client [127.0.0.1]
>Thread-61::DEBUG::2013-11-26 14:10:48,235::task::579::TaskManager.Task::(_updateState) Task=`a89df2de-204c-4aa6-ad3f-c3eecee0058c`::moving from state init -> state preparing
>Thread-61::INFO::2013-11-26 14:10:48,235::logUtils::44::dispatcher::(wrapper) Run and protect: uploadVolume(sdUUID='23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', spUUID='4f04507e-5611-4853-91ff-4384cabd8f34', imgUUID='9fe1b677-08cb-4f35-8eec-d22e199b357f', volUUID='e1f754c5-49b1-4d6f-a738-5ed24f712ee9', srcPath='/tmp/tmp2aGspO', size='20', method='rsync', options=None)
>Thread-61::DEBUG::2013-11-26 14:10:48,236::resourceManager::197::ResourceManager.Request::(__init__) ResName=`Storage.4f04507e-5611-4853-91ff-4384cabd8f34`ReqID=`6d808bda-f53a-465c-8e64-6397a55759fd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1437' at 'uploadVolume'
>Thread-61::DEBUG::2013-11-26 14:10:48,236::resourceManager::541::ResourceManager::(registerResource) Trying to register resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34' for lock type 'shared'
>Thread-61::DEBUG::2013-11-26 14:10:48,237::resourceManager::600::ResourceManager::(registerResource) Resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34' is free. Now locking as 'shared' (1 active user)
>Thread-61::DEBUG::2013-11-26 14:10:48,237::resourceManager::237::ResourceManager.Request::(grant) ResName=`Storage.4f04507e-5611-4853-91ff-4384cabd8f34`ReqID=`6d808bda-f53a-465c-8e64-6397a55759fd`::Granted request
>Thread-61::DEBUG::2013-11-26 14:10:48,237::task::811::TaskManager.Task::(resourceAcquired) Task=`a89df2de-204c-4aa6-ad3f-c3eecee0058c`::_resourcesAcquired: Storage.4f04507e-5611-4853-91ff-4384cabd8f34 (shared)
>Thread-61::DEBUG::2013-11-26 14:10:48,238::task::974::TaskManager.Task::(_decref) Task=`a89df2de-204c-4aa6-ad3f-c3eecee0058c`::ref 1 aborting False
>Thread-61::DEBUG::2013-11-26 14:10:48,241::fileVolume::528::Storage.Volume::(validateVolumePath) validate path for e1f754c5-49b1-4d6f-a738-5ed24f712ee9
>Thread-61::INFO::2013-11-26 14:10:48,262::volume::793::Storage.Volume::(prepare) Volume: preparing volume 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/e1f754c5-49b1-4d6f-a738-5ed24f712ee9
>Thread-31::DEBUG::2013-11-26 14:10:50,849::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:10:51,245::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000496286 s, 1.0 MB/s\n'; <rc> = 0
>Thread-61::DEBUG::2013-11-26 14:10:53,116::sp::2009::Storage.Misc.excCmd::(uploadVolume) '/usr/bin/rsync -aq /tmp/tmp2aGspO /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/images/9fe1b677-08cb-4f35-8eec-d22e199b357f/e1f754c5-49b1-4d6f-a738-5ed24f712ee9' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:11:01,350::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:11:01,423::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.0314387 s, 16.1 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:11:11,515::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:11:11,523::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000311518 s, 1.6 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:11:21,540::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:11:21,547::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000386147 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:11:31,802::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:11:32,580::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.112187 s, 4.5 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:11:42,779::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:11:42,966::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.0512811 s, 9.8 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:11:53,020::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:11:53,100::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.0728573 s, 6.9 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:12:03,104::domainMonitor::186::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-31::DEBUG::2013-11-26 14:12:03,154::fileSD::154::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-31::DEBUG::2013-11-26 14:12:03,194::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend
>Thread-31::DEBUG::2013-11-26 14:12:03,204::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=4f04507e-5611-4853-91ff-4384cabd8f34', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine', 'ROLE=Master', 'SDUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f301ce975088079aedbb732b24ddd897ad868d4d']
>Thread-31::DEBUG::2013-11-26 14:12:03,207::fileSD::572::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images []
>Thread-31::INFO::2013-11-26 14:12:03,207::sd::374::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_imageNS already registered
>Thread-31::INFO::2013-11-26 14:12:03,207::sd::382::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_volumeNS already registered
>Thread-31::DEBUG::2013-11-26 14:12:03,221::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:12:03,228::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000298737 s, 1.7 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:12:13,325::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:12:13,390::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.0343595 s, 14.7 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:12:23,518::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:12:23,637::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.0448552 s, 11.3 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:12:34,027::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:12:34,153::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.0367476 s, 13.7 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:12:44,348::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:12:44,541::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.143355 s, 3.5 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:12:54,631::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:12:55,445::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.0529648 s, 9.5 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:13:05,545::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:13:05,637::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000368914 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:13:16,390::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:13:17,375::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.0481899 s, 10.5 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:13:27,517::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:13:27,874::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.318234 s, 1.6 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:13:38,669::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:13:39,060::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.0457016 s, 11.0 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:13:49,476::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:13:49,551::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.0364323 s, 13.9 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:13:59,642::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:13:59,754::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.0448695 s, 11.3 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:14:09,804::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:14:10,073::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000313405 s, 1.6 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:14:20,138::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:14:20,300::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.124884 s, 4.0 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:14:30,442::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:14:30,561::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.0452309 s, 11.2 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:14:40,802::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:14:40,932::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.0448028 s, 11.3 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:14:51,204::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:14:51,304::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.0448938 s, 11.2 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:15:01,402::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:15:01,476::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.0366037 s, 13.8 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:15:11,583::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:15:11,710::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.0614894 s, 8.2 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:15:21,801::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:15:22,235::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.0448754 s, 11.3 kB/s\n'; <rc> = 0
>Thread-61::DEBUG::2013-11-26 14:15:23,998::sp::2009::Storage.Misc.excCmd::(uploadVolume) SUCCESS: <err> = ''; <rc> = 0
>Thread-61::INFO::2013-11-26 14:15:23,998::logUtils::47::dispatcher::(wrapper) Run and protect: uploadVolume, Return response: None
>Thread-61::DEBUG::2013-11-26 14:15:23,999::task::1168::TaskManager.Task::(prepare) Task=`a89df2de-204c-4aa6-ad3f-c3eecee0058c`::finished: None
>Thread-61::DEBUG::2013-11-26 14:15:23,999::task::579::TaskManager.Task::(_updateState) Task=`a89df2de-204c-4aa6-ad3f-c3eecee0058c`::moving from state preparing -> state finished
>Thread-61::DEBUG::2013-11-26 14:15:23,999::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4f04507e-5611-4853-91ff-4384cabd8f34': < ResourceRef 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34', isValid: 'True' obj: 'None'>}
>Thread-61::DEBUG::2013-11-26 14:15:24,000::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-61::DEBUG::2013-11-26 14:15:24,000::resourceManager::615::ResourceManager::(releaseResource) Trying to release resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34'
>Thread-61::DEBUG::2013-11-26 14:15:24,000::resourceManager::634::ResourceManager::(releaseResource) Released resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34' (0 active users)
>Thread-61::DEBUG::2013-11-26 14:15:24,001::resourceManager::640::ResourceManager::(releaseResource) Resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34' is free, finding out if anyone is waiting for it.
>Thread-61::DEBUG::2013-11-26 14:15:24,001::resourceManager::648::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4f04507e-5611-4853-91ff-4384cabd8f34', Clearing records.
>Thread-61::DEBUG::2013-11-26 14:15:24,001::task::974::TaskManager.Task::(_decref) Task=`a89df2de-204c-4aa6-ad3f-c3eecee0058c`::ref 0 aborting False
>Thread-61::ERROR::2013-11-26 14:15:24,019::SecureXMLRPCServer::168::root::(handle_error) client ('127.0.0.1', 36387)
>Traceback (most recent call last):
>  File "/usr/lib64/python2.6/SocketServer.py", line 570, in process_request_thread
>    self.finish_request(request, client_address)
>  File "/usr/lib64/python2.6/SocketServer.py", line 332, in finish_request
>    self.RequestHandlerClass(request, client_address, self)
>  File "/usr/lib64/python2.6/SocketServer.py", line 627, in __init__
>    self.handle()
>  File "/usr/lib64/python2.6/BaseHTTPServer.py", line 329, in handle
>    self.handle_one_request()
>  File "/usr/lib64/python2.6/BaseHTTPServer.py", line 323, in handle_one_request
>    method()
>  File "/usr/lib64/python2.6/SimpleXMLRPCServer.py", line 490, in do_POST
>    self.send_response(200)
>  File "/usr/lib64/python2.6/BaseHTTPServer.py", line 384, in send_response
>    self.send_header('Server', self.version_string())
>  File "/usr/lib64/python2.6/BaseHTTPServer.py", line 390, in send_header
>    self.wfile.write("%s: %s\r\n" % (keyword, value))
>  File "/usr/lib64/python2.6/socket.py", line 324, in write
>    self.flush()
>  File "/usr/lib64/python2.6/socket.py", line 303, in flush
>    self._sock.sendall(buffer(data, write_offset, buffer_size))
>  File "/usr/lib64/python2.6/site-packages/M2Crypto/SSL/Connection.py", line 222, in write
>    return self._write_bio(data)
>  File "/usr/lib64/python2.6/site-packages/M2Crypto/SSL/Connection.py", line 205, in _write_bio
>    return m2.ssl_write(self.ssl, data, self._timeout)
>SSLError: (32, 'Broken pipe')
>Thread-31::DEBUG::2013-11-26 14:15:32,304::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:15:32,312::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000405861 s, 1.2 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:15:42,331::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:15:42,339::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000383151 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:15:52,363::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:15:52,371::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000469505 s, 1.1 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:16:02,391::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:16:02,401::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000443217 s, 1.1 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:16:12,418::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:16:12,425::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000403879 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:16:22,441::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:16:22,448::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000395308 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:16:32,465::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:16:32,472::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000446578 s, 1.1 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:16:42,490::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:16:42,498::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000341192 s, 1.5 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:16:52,515::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:16:52,522::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000461483 s, 1.1 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:17:02,538::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:17:02,546::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000438891 s, 1.2 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:17:12,550::domainMonitor::186::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-31::DEBUG::2013-11-26 14:17:12,557::fileSD::154::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-31::DEBUG::2013-11-26 14:17:12,559::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend
>Thread-31::DEBUG::2013-11-26 14:17:12,569::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=4f04507e-5611-4853-91ff-4384cabd8f34', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine', 'ROLE=Master', 'SDUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f301ce975088079aedbb732b24ddd897ad868d4d']
>Thread-31::DEBUG::2013-11-26 14:17:12,571::fileSD::572::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images []
>Thread-31::INFO::2013-11-26 14:17:12,571::sd::374::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_imageNS already registered
>Thread-31::INFO::2013-11-26 14:17:12,571::sd::382::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_volumeNS already registered
>Thread-31::DEBUG::2013-11-26 14:17:12,583::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:17:12,590::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000442574 s, 1.1 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:17:22,606::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:17:22,613::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000443448 s, 1.1 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:17:32,629::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:17:32,637::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000319826 s, 1.6 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:17:42,659::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:17:42,667::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000285148 s, 1.8 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:17:52,687::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:17:52,695::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000419912 s, 1.2 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:18:02,712::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:18:02,720::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000436487 s, 1.2 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:18:12,736::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:18:12,743::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000319359 s, 1.6 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:18:22,759::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:18:22,766::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000529283 s, 954 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:18:32,783::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:18:32,790::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.00034487 s, 1.5 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:18:42,807::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:18:42,815::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000398134 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:18:52,832::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:18:52,840::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000515216 s, 980 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:19:02,856::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:19:02,863::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000330751 s, 1.5 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:19:12,882::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:19:12,891::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000447842 s, 1.1 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:19:22,907::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:19:22,915::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000435854 s, 1.2 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:19:32,930::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:19:32,938::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000329755 s, 1.5 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:19:42,961::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:19:42,989::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.00789573 s, 64.0 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:19:53,013::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:19:53,021::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000349814 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:20:03,042::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:20:03,049::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000324344 s, 1.6 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:20:13,066::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:20:13,073::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000347428 s, 1.5 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:20:23,090::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:20:23,097::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000358646 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:20:33,113::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:20:33,121::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000419091 s, 1.2 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:20:43,138::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:20:43,145::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000337929 s, 1.5 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:20:53,162::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:20:53,170::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000408984 s, 1.2 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:21:03,335::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:21:03,420::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000366908 s, 1.4 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:21:13,540::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:21:13,548::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000376467 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:21:23,564::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:21:23,572::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.00039612 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:21:33,588::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:21:33,595::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000446084 s, 1.1 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:21:43,612::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:21:43,620::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000447106 s, 1.1 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:21:53,655::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:21:53,668::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000783993 s, 644 kB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:22:03,704::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:22:03,712::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000393037 s, 1.3 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:22:13,725::domainMonitor::186::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-31::DEBUG::2013-11-26 14:22:13,735::fileSD::154::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c
>Thread-31::DEBUG::2013-11-26 14:22:13,737::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend
>Thread-31::DEBUG::2013-11-26 14:22:13,747::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=4f04507e-5611-4853-91ff-4384cabd8f34', 'REMOTE_PATH=10.34.63.202:/mnt/export/nfs/lv3/lsvaty/hosted_engine', 'ROLE=Master', 'SDUUID=23b22ad4-6d4c-4cdc-b25e-6e9efef7004c', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f301ce975088079aedbb732b24ddd897ad868d4d']
>Thread-31::DEBUG::2013-11-26 14:22:13,749::fileSD::572::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images []
>Thread-31::INFO::2013-11-26 14:22:13,749::sd::374::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_imageNS already registered
>Thread-31::INFO::2013-11-26 14:22:13,749::sd::382::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 23b22ad4-6d4c-4cdc-b25e-6e9efef7004c_volumeNS already registered
>Thread-31::DEBUG::2013-11-26 14:22:13,760::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:22:13,768::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.00041921 s, 1.2 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:22:23,792::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:22:23,800::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000448971 s, 1.1 MB/s\n'; <rc> = 0
>Thread-31::DEBUG::2013-11-26 14:22:33,821::fileSD::239::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_lsvaty_hosted__engine/23b22ad4-6d4c-4cdc-b25e-6e9efef7004c/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-31::DEBUG::2013-11-26 14:22:33,831::fileSD::239::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n505 bytes (505 B) copied, 0.000375085 s, 1.3 MB/s\n'; <rc> = 0

Actions: View

Attachments on bug 1034777: 829270 | 829271